We took great care when setting up and running benchmarks to be sure we were gathering data that could be used to make useful comparisons. We wanted to be able to compare the ZFSBuild2012 design with the ZFSBuild2010 design. We also wanted to be able to compare various configuration options within the ZFSBuild2012 design, so we could make educated decisions about how to configure a variety of options to get the most performance out of the design. The purpose of this page is to share all of our benchmarking methods.
We used the same blade server that we used for the ZFSBuild2010 testing back in 2010. The specs for that blade are available at http://www.zfsbuild.com/2010/05/05/test-blade-configuration/ . Note, we did not merely use a blade with the same specs. We literally removed that exact blade server from production and reassigned it back to our test lab. We did install software updates (OS, drivers, OFED, etc) and a firmware update in the InfiniBand card. We did not make any hardware changes to the test blade server since we run the benchmarks in 2010. The test blade server was running Windows 2008R2, just like it was for the tests back in 2010.
All of the benchmarks were run on the test blade server using IOMeter. We used the same IOMeter configuration file that we ran back in 2010. That configuration file is available at http://www.zfsbuild.com/pics/Graphs/Iometer-config-file.zip . The only change we made in that file was the disk target (replacing “D:New Volume” with the correct path).
The benchmarks are run over the wire. IOMeter runs on the Windows 2008R2 test blade server, and connects over the network to the ZFSBuild2012 server. In every test, the test blade and the ZFSBuild2012 server are plugged into the same network switches. In some cases this is 1Gbps Ethernet. In others, it is 20Gbps InfiniBand.
We did not use multipathing during any of the benchmarks. In fact, we took it a step further and disabled every NIC that we were not using, so there was no chance packets would go down the wrong path.
All of our testing focuses on the SAN capabilities, not the NAS capabilities. This means we are testing iSCSI or SRP instead of NFS or CFS. We realize that some people are interested in seeing the performance of a NAS configuration, but that was beyond the scope of our tests.
All of our tests were on a RAID10 style ZFS pool. We created a ZFS pool called “BigDog”. The BigDog pool contained 18 1TB SAS drives arranged into 9 mirrored pairs, two 1TB SAS drives as hot spares, two Intel 520 series 240GB SSD drives as L2ARC, and two Intel 313 series 20GB SSD drives as mirrored ZIL drives. All of our tests were run using ZVols created in this “BigDog” ZFS pool. When we switched to different storage appliance software, we imported the entire “BigDog” ZFS pool rather than reconfiguring it from scratch each time. This way we could test the exact same ZFS pool with multiple storage appliance software packages using the exact same hardware. When switching between storage appliance software, we only changed the contents of the boot drives (the two internal Intel 330 series 60GB SSD drives).
We deleted the IOMeter test file in between each round of benchmarks, and rebooted the test blade server and ZFSBuild2012 box. We did this to make sure caching from previous benchmark runs did not impact the next benchmark run.
The software versions we used during testing were Nexenta 3.1.3, ZFSGuru 0.2.0-beta 7 (FreeBSD 9.1), and FreeNAS 8.3 (FreeBSD 8.3). We realize that newer versions have been released since we ran our benchmarks. We will not be re-running any of the benchmarks at this time, because we have already placed the ZFSBuild2012 server into production. The ZFSBuild2012 server has spent more than a month as a high performance InfiniBand SRP target for one of our web hosting clusters.
Similarly, the ZFSBuild2010 is also in production, so we could not re-run any benchmarks using the old ZFSBuild2010 box with newer Nexenta builds. Any references to the performance of the ZFSBuild2010 design will be based on the hardware and software in the ZFSBuild2010 box when the original articles were written back in 2010.
If you have any questions about benchmark methods for the ZFSBuild2012 project, please ask them here. We will be happy to add more information to this page to better explain our testing methods.
9 Comments to ZFSBuild2012 – Benchmark Methods
Leave a Reply
You must be logged in to post a comment.