The Pi case arrived and has now been assembled.
There isn't much room from one Pi to the layer above and the Pi's were reporting temps in the 60's (degrees C). I added a fan on the side to help move the warm air from each layer which has reduced the temperature to the mid to high 50 degrees.
I still need to get short network cables to help tidy things up.
The Pi5's have now passed the RAC (Recent Average Credit) that the Pi4's were getting so they are being more productive.
Marks Rpi Cluster continues to run 24/7.
Seeing as the Edge Cluster from BitScope isn't available for the Pi5 I purchased four Pi5's and associated bits to add them to the cluster.
4 x Raspberry Pi5 (4GB)
4 x Raspberry Pi Active Cooler
4 x Raspberry Pi5 USB-C Power Supply
5 x Sandisk 32GB High Endurance micro SD cards
1 x Power board with 8 outlets
I ordered one of the stackable acrylic cases from eBay as I couldn't find them available from the usual Rpi shops. They are just sheets of clear acrylic and a bunch of metal stand-offs. While I wait for the case I have them sitting loose on an old TV table. I need to buy some short network cables as I used a bunch of 2 meter ones and its like spaghetti junction at the moment. I had to get a power board (aka a power strip) due to all the power supplies. I had a spare 8 port switch which I used and the network cables.
You can see the four Pi5's just sitting on the table with their active coolers. Thats an 8 port switch on the top right with a Pi4 sitting on top of it. Is got an external HDD plugged in. On the bottom left are two more Pi4's. The one with the blue thing on top is a proxy server/Pihole machine and not part of the cluster. The blue thing is a Samsung T5 portable SSD. The Pi in the bottom left corner is a time server. The beige USB cable is plugged into a GPS.
Hopefully the stackable case will arrive soon so I can tidy things up. My aim was to get them going as quickly as possible. The delivery date on the case is supposedly the 29th of June.
The Pi5's are running Asteroids and Einstein work. They have shaved a few hours off for the Asteroids Period Search app. Einstein BRP4 on a Pi4 typically take 4 hours 40 minutes and on the Pi5 seem to be 2 hours 52 minutes (running 4 at a time).
The cluster now consists of:
4 x Pi5 (4GB) compute nodes.
24 x Pi4 (8GB) compute nodes.
Plus a Pi5 and two Pi4 as support nodes.
Marks Rpi Cluster continues to run 24/7. Its has been running Einstein BRP4 work for the last couple of months.
Last weekend the project had an issue and all file transfers were suspended. According to their message boards they disabled the transitioner. When I checked on the farm I noticed lots of file transfers stuck and only the remaining work finishing off. At that point after retrying a few transfers, which were still stuck, I decided to let the farm run Asteroids work. Come the Monday and the file transfers had cleared up and we're back to running BRP4.
This week one of the Pi4's locked up. In order to reboot I have to run off all work and power cycle the edge cluster. So I suspend any work waiting to start and let it finish off any that are already running. Come back 5 hours later and tell each node to shut down before turning the power off and on again. I don't know what caused this node to lock up but suspect the micro SD card is probably worn out. To change them I would have to open up the Edge Cluster find the particular node and replace the SD card and then reinstall things. At the moment its back after the power cycle so I will see if it has any further issues.
You can run Pis using network storage, but I don't, I took the simpler solution of using the micro SD card for booting and local storage.
I have a couple of Pi's with USB connected SSD's. One is a Pi4 with a Samsung T5 and the other is a Pi5 with a Samsung T7. If you try to run fstrim on these if doesn't do anything, even though both devices support trim. After a bit of google-foo I managed to find a solution.
Identify vendor/product ids
First you'll need to identify the vendor id and product id. You can do this with the lsusb command. It will give you two hex values for each device. The first is the vendor and the second is the product id.
$ lsusb
Bus 004 Device 002: ID 04e8:4001 Samsung Electronics Co., Ltd PSSD T7
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
In this case you can see the vendor is 04e8 and the product id is 4001 for my T7.
The same command on my Pi4 with a T5...
$ lsusb
Bus 002 Device 002: ID 04e8:61f5 Samsung Electronics Co., Ltd Portable SSD T5
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
In this case you can see the vendor is 04e8 and the product id is 61f5 for my T5.
Create udev rule
Now we have the two ids we can create a udev rule to tell it that trim is supported by the device. We need to add the rule by editing a file.
$ sudo nano /etc/udev/rules.d/55-usb-ssd-trim.rules
And then paste in the following (for a T7). Change the idProduct if you have a T5.
ACTION=="add|change",ATTRS{idVendor}=="04e8", ATTRS{idProduct}=="4001", SUBSYSTEM=="scsi_disk",ATTR{provisioning_mode}="unmap"
Save this (Ctrl-O) and exit (Ctrl-X).
I believe you have to reboot for it to pickup the new rule but I could be wrong here. I rebooted anyway. After this you should be able to run fstrim.
$ sudo fstrim -av
/boot/firmware: 446.9 MiB (468604928 bytes) trimmed on /dev/sda1
/: 0 B (0 bytes) trimmed on /dev/sda2
Check fstrim.timer
You can also check the fstrim.timer service is running. It will usually run fstrim once a week.
$ sudo systemctl status fstrim.timer
fstrim.timer - Discard unused blocks once a week
Loaded: loaded (/lib/systemd/system/fstrim.timer; enabled; preset: enabled)
Active: active (waiting) since Sat 2024-06-08 18:29:50 AEST; 3h 23min ago
Trigger: Mon 2024-06-10 00:23:17 AEST; 1 day 2h left
Triggers: * fstrim.service
Docs: man:fstrim
Press Q to exit from systemctl.
You can see its waiting until Monday at 0:23AM before it will run fstrim.
