ZFSBuild2012 – Specs and Parts Pictures
We took a lot of pictures while we were building and testing the ZFSBuild2012 SAN.
The ZFSBuild 2012 system is comprised of the following :
SuperMicro SC846BE16-R920 chassis – 24 bays, single expander, 6Gbit SAS capable.
SuperMicro X9SRI-3F-B Motherboard – Single socket Xeon E5 compatible motherboard.
Intel Xeon E5 1620 – 3.6Ghz latest generation Intel Xeon CPU.
20x Toshiba MK1001TRKB 1TB SAS 6Gbit HDD’s – 1TB SAS drives.
LSI 9211-8i SAS controller – Moving the SAS duties to a Nexenta HSL certified SAS controller.
Intel SSD’s all around
2x Intel 313 series 20GB SSD drives for ZIL
2x Intel 520 series 240GB SSD drives for L2ARC
2x Intel 330 series 60GB SSD drives for boot (installed into internal cage)
64GB RAM (8x 8GB) – Generic Kingston ValueRAM.
20Gbps ConnectX InfiniBand card
4x ICYDock 2.5″ to 3.5″ drive bay converters
Internal drive bay bracket for boot drives
Y-cable for the rear fans (not enough fan sockets without one Y-cable)
Here are some pictures of the first few parts that showed up:




Here are pictures of the box the chassis came in:





Here are pictures of the SSD Drives:




Here are pictures of the box of 1TB SAS drives:










Here are pictures of the motherboard:















Here are pictures of the CPU Heatsink:



Here are pictures of the fan Y-cable:





Here are pictures of the InfiniBand Card:




Here are pictures of the ICY Dock 2.5″ to 3.5″ converter:




Here are pictures of the Internal drive bay bracket for boot drives:



9 Comments to ZFSBuild2012 – Specs and Parts Pictures
Leave a Reply
You must be logged in to post a comment.
Hi,
Great system! Quick question: Nexenta doesn’t recommend using SATA disks with a SAS expander, and the SSDs are SATA disks. Have you noticed any issues with this setup? We are about to start another server build, using the 847E16-JBOD1 for the disk subsystem, and are in the process of sourcing L2ARC and ZIL drives, so would appreciate any feedback. If we have the budget, the ZIL will be a ZeusRAM, but still need L2ARC. We were thinking about installed the L2ARC disks in the server itself.
Thanks,
Kevin
So your caching SSDs are sata. Did you have to get any adapters for that? How are those presented to you in nexenta? Just as another cNNWWID device ?
Hi Matt – Great blog thanks.
Which IB HBA ConnectX did you use? I’ve heard the cards with no on board memory aren’t supported in Nexenta.
We used the MHRH29-XTC. I do not know that there are _any_ IB cards on the HSL. For the InfinihostIII cards, that is true. For the ConnectX cards, you do not need memory on the card from what I can tell. They use different Drivers (Tavor vs Hermon iirc) and the Connectx stuff seems to work significantly better than the Infinihost based cards.
Thanks for clarifying – didn’t realise it only applied to InifiHostIII cards.
Question regarding the LSI 9211-8i SAS, do you have to configure each drive as a single drive within the cards bios? or do you just attach drives and they are instantly provided to the operating system?
Thanks!
Since this is simply an HBA, rather than a RAID card, we do not have to configure each individual drive in the card bios, they are simply presented to the OS as individual drives. It is recommended to do this rather than use a RAID card and configure each drive as RAID cards typically mask some of the underlying drive behavior which is not desirable in ZFS.
[...] [...]
The case is very nice. I started digging around a bit and found a case/backplane combo that is interesting. It doesn’t seem like a real backplane but thermaltake makes these x6 2.5″ drive per 5.25″ bay “things” that would allow for up to 72 2.5″ hotswappable drives in this arguably bush-league case. The reliability of the thermaltake is questionable. The case has 12 5.25″ external bays and room for 2 PSUs and a LOT of fans, haha. Would have to drop $5k or more on that many drives, but this could maybe provide a feasible 60-70TB JBOD to work with for 5400rpm 8mb archival action.
Xigmatek Elysium – CCC-HSA0DS-U01 – ATX Super Tower – $169
http://www.newegg.com/Product/Product.aspx?Item=N82E16811815011
12 x thermaltake MAX-1562 backplane $900
http://www.amazon.com/Thermaltake-Backplane-Removable-2-5-Inches-RC1600101A/dp/B004G8QERK/ref=sr_1_cc_1?s=aps&ie=UTF8&qid=1361125237&sr=1-1-catcorr&keywords=2.5+thermaltake+backplane