A couple of months ago I decided it was time to fulfil a childhood dream and aquire something I’d always wanted. Not a sports car, or a girlfriend 20 years younger than me…..no, my midlife crisis would be embodied by a server rack. Here then is the tale of an idiot building a server rack. Because why not?
First, I needed the hardware to go in it. I managed to snag a pair of HP DL380s on ebay for the princely sum of £75 (for the pair), a dell r810 (dual 8 core processors, noice) for another £50 and an incredibly old Xyratek Fibre SAN enclosure fully populated with 12 SAS drives for £100.
First off….. those DL380s. They were advertised with 16GB each. When I got them home (after a 4 hour round trip) and booted them, they actually had 6GB in. Grr. After some back and forth with the seller (he was actually a nice guy who seemed to have been done wrong by his IT provider) I got a £30 refund which I promptly spent on more memory to bring these up to 12GB each and the Dell up to 64GB. I plan on running any VMs I need on the dell, so bunging more memory in it makes sense at this stage, and I can add more memory to the HPs as and when needed.
That fibre san was…an adventure. It’s an RS-1220-X RAID which was state of the art in like, 2005 or something. Xyratex has changed hands several times since then, eventually being owned by seagate who shut down the whole division in 2011. Finding manuals, drivers etc for this was challenging. However, I eventually found a forum post by someone selling one of these units 5 years ago(!) and he happened to mention that he’d include all the drivers, firmware etc that he’d collected for them. He also used his full name in the post, which lead to me looking him up on linkedin. Luckily he was a really nice guy, who happily sent me over a zip file containing just about every manual, firmware etc that ever existed for these. Which you can get here.
After all that, I happily created a RAID6 array on the thing, hooked up my server via a fibre card and…. was incredibly dissapointed in the write speeds. Apparently 2006 hardware was not great at calculating parity on the fly. So back to the drawing board.
I did some digging around on ebay, and managed to snag a netapp ds-4243 SAS array for £60. 24 bays, 3gb SAS connectivity (upgradeable to 6gb by swapping out the controllers). Very nice indeed. After some unfortunate messing around with cables and cards I was in business. Note for anyone getting one of these, the array has QSFP+ ports on it. Just about every SAS card you can buy has mini-SAS ports on it….except netapp’s own SAS card. However, Netapp’s own SAS card is complete junk, do yourself a favour and get yourself a QSFP+ to mini-sas cable and a standard SAS card. You’ll save yourself a lot of hassle.
Note that the one in the pictrue there has drive trays. Mine did not. However, the ones from the Xyratex fit straight into it. Score! Apparently Xyratex made a lot of enclosures for other companies back in the day, including netapp. I have 12 drive trays from the Xyratex and I can print some more if I need them
So, I was now getting somewhere. I had 3 servers. I had storage. Now for networking. I decided to keep things simple (although I had a brief foray into the world of infiniband. 40gb/s switch for about £80? nice) and just go with a 24 port procurve gigabit switch. This also has 4 fibre ports in it, so I can stick my 3 servers in via 8gb connections if I need to at some point but for now I’m just hooking them up with gigabit. For routing I like OPNsense, a fork of the venerable pfsense. I originally had this running on a mini-itx board just lying around case free which was fine, but if I’m building a rack why not rack mount everything? Then I saw a watchguard XTM-5 series on ebay. These are a bit long in the tooth now, but perfect for running OPNSense on (and you can upgrade the processor & memory in them). It was listed as an auction, so I thought I’d be a bit cheeky:
Oh well, I tried, guess I’ll just have to bid on it instead…..
Anyway, that was servers, storage and networking sorted out, now I just needed a home for all this kit. I needed a rack that had some measure of noise reduction (since it’s going to be about a metre away from where I work), about 14u or so and on wheels so I could move it about easily. eBay wasn’t turning anything up for less than £stupid so what to do? Then I noticed this in the corner of the cellar…..
(not the lego. the spare bits of wood / old furniture). I wonder if…….