We have received a lot of feedback from members of the IT community since we published our benchmarks comparing OpenSolaris and Nexenta with an off the shelf Promise VTrak M610i. One question we received from several people was about FreeNAS. Several people asked “How does FreeNAS compare to OpenSolaris on the same hardware?” That was an excellent question, and we decided to run some tests to answer that question.
NOTE: This article was written in 2010 using FreeNAS 0.7.1. We have a new article with much newer benchmarks from 2012 posted about FreeNAS 8.3 at http://www.zfsbuild.com/2013/01/25/zfsbuild2012-nexenta-vs-freenas-vs-zfsguru/. We strongly encourage you to read both this article and the newer article before passing judgement on FreeNAS, because there were significant performance improvements made in FreeNAS in recent versions.
FreeNAS makes it really easy to deploy a SAN or NAS based on FreeBSD. FreeBSD has ZFS support, but it is an older version of ZFS than what is available in OpenSolaris. The ZFS version can be a big deal. For example, the version of ZFS included with FreeBSD does not include support for deduplication, so you cannot use deduplication with FreeNAS.
We like the web based GUI included with FreeNAS. It makes it relatively easy to configure a FreeNAS box. Not everything works properly in the GUI, though. We had problems creating ZFS pools using the GUI. We finally reverted to using the command line to build the ZFS pool (zpool create and zpool add commands). Once we had the ZFS set up, we told the GUI to synchronize the GUI with the ZFS pool, so we could see ZFS information through the GUI. That worked, but it would have been more pleasant if the GUI had been able to create the ZFS pool without the need for us to revert to the command line.
The next issue we ran into we was an oversight in the GUI. With the zpool command, you can add devices to an existing ZFS pool. The GUI does not have any way of handling that. To add devices to an existing ZFS pool, you need to use the zpool add command and then the sync option in the FreeNAS GUI.
Once we had the ZFS pool completely configured, we set up an iSCSI target through the FreeNAS GUI. Unfortunately, you cannot directly share a ZFS volume using iSCSI with FreeNAS. The GUI has an option for doing that, but the GUI does not list any ZFS volumes in the drop down box. The only way to successfully share a ZFS volume via iSCSI is if you create a file extent on the ZFS volume and then share the iSCSI target using that file instead of the ZFS volume directly. This was not terribly difficult to do, but it was definitely not obvious. At the very least, the GUI should be smart enough to share a ZFS volume by creating the file based extent behind the scenes.
I realize that not a lot of people have an InfiniBand network, but I feel I should touch on this issue anyway since we do. In many cases, you can deploy a 20Gbps InfiniBand network for less than you can deploy a 10Gbps Ethernet network. Unfortunately, FreeBSD does not have any support for InfiniBand, so you cannot use InfiniBand at all with FreeNAS. It is a shame, because InfiniBand offers outstanding performance for the price.
When we tested FreeNAS with a ZFS volume, we noticed that FreeNAS could see all 12GB of memory, but FreeNAS only chose to use 4GB. This limited how much ARC caching FreeNAS could do. With OpenSolaris, all of the memory above the first 1GB of memory is used for the ARC caching. This means OpenSolaris was able to use 11GB for caching, which was clearly better than the 4GB that FreeNAS limited itself to. We were definitely using the 64 bit version of FreeNAS, so the issue was not a 32 bit limitation.
Throughout all of our testing with FreeNAS, one thing kept annoying us. FreeNAS would sometimes forget the IP on boot. When this happened, we had to use IPMI to remote KVM into the box and manually configure the network settings from the command line.
We used exactly the same hardware for the FreeNAS box as we used for the OpenSolaris and Nexenta testing. We also used the same test blade to run Iometer. This means our OpenSolaris and Nexenta results should be able to be compared with FreeNAS. Additionally, we set up exactly the same ZFS volume on FreeNAS. The ZFS volume was 18 active drives in RAID, plus SLC drives for ZIL, two MLC drives for L2ARC caching, and two hard drives for spares. We used the same 24 drives that we used in the OpenSolaris and Nexenta benchmarks.
Here is the actual IOmeter config file:
As with our previous benchmarks, we use 1Gbps Ethernet to connect the test blade with the SAN box. We also have 20Gbps InfiniBand connected to all of the hardware, but none of these benchmarks used InfiniBand. We set up an iSCSI target on FreeNAS and ran Iometer from the test blade using the exact same list of tests we used for our previous benchmark article.
Once all of the tests completed, we also tore down the iSCSI target and ZFS. Then we set up a FreeNAS Software RAID volume and an iSCSI target pointing to the Software RAID volume. The software RAID volume was configured as a RAID10 volume with 18 drives, since that was the core of each ZFS volume we had constructed so far for testing. Then we ran all of our benchmarks against the Software RAID based iSCSI target on the FreeNAS box. In this benchmark section, we will be comparing OpenSolaris with both FreeNAS/ZFS and FreeNAS/SoftwareRAID.
4k Benchmark Results:
8k Benchmark Results:
16k Benchmark Results:
32k Benchmark Results:
We don’t really understand why FreeNAS performed so poorly compared to OpenSolaris using exactly the same hardware and test environment. In some tests, the FreeNAS with ZFS performed even worse than a software RAID volume did in FreeNAS. Given the results, we cannot recommend using FreeNAS for enterprise grade file storage. We expected FreeNAS to perform much better than it actually did. We were amazed by how slow FreeNAS was compared to OpenSolaris.
But that does not mean everybody should stop using FreeNAS. There are times when FreeNAS is a great solution. For a low end file server at home, it is probably fine. One thing FreeNAS does do well is open up a lot of different file sharing methods. For example, FreeNAS can directly share files through FTP or even BitTorrent. This may sound really silly when we are looking for enterprise SAN solutions, but these are the types of things home users will use a file server for. So even though we would never consider FreeNAS for enterprise grade SAN solutions, we can understand why home users might really like FreeNAS.
32 Comments to FreeNAS vs OpenSolaris ZFS Benchmarks
Leave a Reply
You must be logged in to post a comment.