After a bit of break Marks Rpi Cluster is back online. I had building and painting happening so they were powered down while this took place. The cluster is busy again with Einstein and Rosetta work.
I powered up the farm starting with the two support nodes. They came up but for some reason couldn't be found on the network. I tracked this down to an issue with the power adapter for the switch that they were connected to. Once fixed I brought up the remaining Pi4's without any further incidents.
The following day I tried to get the Pi3's going and none of them would work. I had to plug a screen and keyboard into one to see what was happening. It seems all 6 of them had corrupt SD cards. I ended up having to re-image them and reinstall each one which took a little time. They're now running Einstein work.
I've ordered another Pi4 (2GB) to take over the time keeping role. At the moment the storage server Pi also has a GPS plugged in to provide time. I will move it over to this dedicated Pi, once it arrives. GPS's send strings of data every second that is interpreted and passed onto the NTP client. Its a constant CPU load (around 2-5% on a Pi4).
I will be chasing up the 3D printing studio regarding the revised Pi case tomorrow to see if I can get them sorted out. The last contact I had was on the 13th of January saying they would have the design finalised in a week. Its now been a fortnight.