couteausuis.se thoughts

bitcoin farm architecture

15 Oct 2022

intro

i’ve been involved in multiple bitcoin mining farms since 2013, currently operating a small, 150 machines farm. i find it’s a nice way to stay up to date with the latest bitcoin developpment, as it involves hardware and software upgrades a few times per year. it’s also a cheap way to obtain bitcoin (cost of production is less than market price) and a nice place to test new ideas (heating, immersion mining, etc).

like when i started in 2013, i’m still offering my services as a consultant for all your bitcoin mining needs.

bitcoin mining

if you have no idea what is mining bitcoin, i invite you to read on it.

bitcoin mining helps us to acquire bitcoin at a much cheaper rate than the market price and similar to the DCA method. you need a small investment upfront for the setup and the machines (you can buy used), and then you only have to pay for electricity monthly, while producing and stacking sats everyday. of course, mining is extremely inefficent and creates a lot of (wasted) heat, but this is also a pretty nice by-product. using that heat, we can heat up homes during the winter or even heat up some greenhouses, drasticaly (~75%) reducing their energy costs. now if you stack your sats (instead of selling at market price) and wait ~4 years, you’ll even turn a profit on that heat enterprise! for further reading, look at the Arcane article.

since bitcoin mining is inefficent and uses a lot of electricity, farm operators are always looking and the price per kilowatt ($/kw) to help their margin. operators have moved accros continents to save on energy prices. this also pushes the use of renewables energy sources, as they are often the cheapest option. bitcoin mining can also help to offset the high capital expenditure needed to launch a renewable energy project. when you’re building a solar panel farm or hydro dam, you’re building it for the expected energy needs in 10,20,30 years and so it will be oversized at the start. the few starting customer will be paying much higher in the first years and in developping countries it’s not even feasable as customers can only afford the price if the project is fully operational. by feeding the surplus energy to bitcoin mining farms, the whole project can turn a profit quickly and greatly reduce costs for the customers, making renewable energy project cheaper and more accesible.

machines

we started the project in 2013, adding machines whenever we can. we are very lucky that our aging S9 are still profitable. we do have to do a lot of maintenance of the aging machines. out friends ad D-Central have helped us a lot in sourcing and repairing hashingboards and psus. the current farm has quite a funky mix of machines:

monitoring

my current farm is located quite remotely from where i live. Cheap electricity and distance from neighboors were the drive factors to decide it’s location. now since it’s far, i don’t want to have to get there often, that’s why lots of thoughts need to be put into the monitoring and remote access. a good remote monitoring solution should cover these points:

secure remote access

i’m using a simple raspberrypi located on the farm site, running stock Raspbian OS. using tailscale as a VPN enables me to connect easily to the farm (much easier and faster than an OpenVPN or WireGuard tunnel) and since the mining site network is a specific LAN subnet (10.66.10.0/24), i can broadcast it from the raspberrypi (effectively turning it into a VPN router) and browse the whole farm and machines from the comfort of my home. tailscale also links every machines under my account, meaning that i can use my homelab (pomelo) or even a VPS to monitor and log the farm. to install tailscale with routing and broadcasting:

curl -fsSL https://tailscale.com/install.sh | sh
echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p /etc/sysctl.conf
sudo tailscale up --advertise-routes=10.66.10.0/24
#activate route in tailscale web admin

pro tip: since i’m using tailscale MagicDNS to use my pihole dns from every device, all dns queries from the raspberrypi will go through my pihole. because i’ll use the raspberrypi as a stratum mining proxy, i don’t want to add this layer of latency and/or breaking point. adding this line at the end of /etc/dhcpd.conf will force it to use cloudflare dns server : static domain_name_servers=1.1.1.1

monitoring and configuring of individual machines

on every machine, i’m installing Braiins OS+ and the next logical step is to use Braiins’ own FarmProxy to reduce bandwidth usage and latency. with FarmProxy, we can aggregate hashpower and split it between partners/customers whitout having to reconfigure single machines. using Braiins OS+ also enables us to monitor the individual machines (power, temperature, hashingboard) since they each have prometheus endpoints. Braiins OS+ can also be configured using btctools, so we can push configurations in batch and easily scan the network when adding new machines or replacing old ones. to install FarmProxy on the raspberrypi:

#install docker & docker-compose
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker ${USER}
sudo apt-get install libffi-dev libssl-dev
sudo apt install python3-dev
sudo apt-get install -y python3 python3-pip
sudo pip3 install docker-compose
sudo systemctl enable Docker
#reboot

#install FarmProxy
sudo apt update
sudo apt install git
git clone https://github.com/braiins/farm-proxy.git
docker-compose up -d

pro tip: before starting FarmProxy, i removed the prometheus and grafana docker services, since i’l be launching those from my homelab and i want to keep the raspberrypi lightweight.

logs, dashboards and alert system

on my homelab (pomelo), i’ve installed btctools, bos-farm-monitor and bos-toolbox:

#install snap and btctools
sudo snap install core btctools

#install FarmProxy Monitoring only
git clone https://github.com/braiins/bos-farm-monitor.git
#Change list of IP address ranges to scan in ./scan_crontab
docker-compose up -d
#localhost:3000

i also made a bunch of scripts to do various batch operations on the machines. I’m using this setup to update the firmware, under/overclock, change pool, locate faulty machines, etc. first, create a file with each machine’s IP address (one line each), then create a script similar to these. (updating T17 machines to the latest Braiins OS+):

#updating T17 machines to the lastest Braiins OS+ firmware
for IP_ADDRESS in $(cat "ListOfUpdate.csv"); do
	ssh -o HostKeyAlgorithms=+ssh-rsa 
        -o PubkeyAcceptedKeyTypes=+ssh-rsa 
    root@$IP_ADDRESS 'wget -O /tmp/firmware.tar firmware_url
        && sysupgrade /tmp/firmware.tar'
done

#locate machine by maing it flash it's LED lights
for IP_ADDRESS in $(cat "ListOfFailes.csv"); do
    ssh root@$IP_ADDRESS 'miner fault_light on'
done

Related Posts