Nexenta Core Platform Benchmarks

In our benchmarks between OpenSolaris b134, NexentaStor Enterprise, and a Promise VTrack M610i box, we found that OpenSolaris consistently outperformed NexentaStor. We were never really quite sure why, since NexentaStor is based on Nexenta Core Platform and Nexenta Core Platform was based on OpenSolaris b134. We expected NexentaStor to match the performance of OpenSolaris, but it simply did not.

One theory we had for the performance difference was that the web GUI in NexentaStor used enough system memory that NexentaStor had significantly less ARC cache available and was therefore at a performance disadvantage to OpenSolaris. This got us curious about how Nexenta Core Platform would perform relative to OpenSolaris and NexentaStor.

We decided to benchmark Nexenta Core Platform using the same hardware and benchmarks that we have used for all of the previous benchmark runs. The results exceeded our expectations.

4k Benchmark Results:


8k Benchmark Results:


16k Benchmark Results:


32k Benchmark Results:


Keep in mind that Nexenta Core Platform is based on OpenSolaris b134, and NexentaStor is based on Nexenta Core Platform. We are now left wondering why there is such a big performance gap between Nexenta Core Platform and NexentaStor. Nexenta Core Platform is very fast. We would have assumed that NexentaStor Enterprise would be just as fast, since it is Nexenta’s flagship product and it is based on Nexenta Core Platform.

There are some reasons to still choose NexentaStor over Nexenta Core Platform. The web GUI is useful for people who don’t want to use the command line. I personally really like the command line for ZFS, but I can understand why some people would prefer the web GUI.

The other advantage of NexentaStor is considerably more important. With OpenSolaris and Nexenta Core Platform, there is no built-in solution for handling notifications (such as drive failures) or for identifying a failed hard drive with a LED light on the front of the chassis. The only way to add these functions to OpenSolaris and Nexenta Core Platform is through custom scripting. With NexentaStor, these two functions are built-in. That means NexentaStor is ready to use as a SAN or NAS solution the moment you install it. So even though NexentaStor did not win in these performance benchmarks, it is still a very serious contender given its management features.

Saturday, October 9th, 2010 Benchmarks

6 Comments to Nexenta Core Platform Benchmarks

  • kayasoze says:

    Very odd results. Does NexentaStor Enterprise enable a large number of DTrace probes to inform their management dashboards? I do not know the performance impacts of DTrace but I speculate that they could be significant in some circumstances.

  • wvandenberge says:

    Have you considered running your benchmark on the commercial version of Solaris also? Partly insprired by this site I put together a 100TB SAN/ZFS/Nexenta Core system that is performing very well in a very busy prodcution environment. However getting commercial software (like the BackupExec agent) to install is proving to be impossible, so I’m actually looking to migrate to commercial Solaris. Ideas?

  • doc says:

    Did you ever determine why NexentaStor’s performance was so bad? From your latest posts it appears you are now running some flavor of Nexenta, so is it safe to assume they’ve solved the problem?

  • We never determined a root cause for the performance differences. In production, we’ve never seen the workloads demand peak performance out of the disk subsystem, so we’ve never worried about it.

    We do have some sneaking suspicions that the amount of RAM in our system isn’t enough for high workloads like which were presented during the benchmarking. Over the last year we have come across several different bits of knowledge that lead us to believe that significantly more RAM is a very good thing in a ZFS based system.

    Consequently, our next build (coming soon) will have more ram. Lots more ram. Lots and lots and lots more ram.

  • rubendob says:

    Hello people

    firstable, your blog is one of my favorites about ZFS. Great stuff. Then I would like to ask you if there is some way to benchmark nexenta of ZFS filesystem not so deep and hard like your benchmark test with iometer. I mean, something like sysbench or dd or something like that to have a clue your san is working ok. I read dd is not a great stuff to benchmark nexenta or any san.

    What do you think?

    Thanks again

  • rubendob says:

    Hello

    I finally learn some about iometer and now I’m able to benchmark our hardware. Firstable, we are very far from you guys :) Our better test shows 5000 iops (read seq).

    Our last test was with supermicro hardware (8 sata disk + 2 ssd 512 GB for cache + 1 50 ssd for zil + 2 50 ssd mirrored for syspool). Volume of 8 disk with no ZFS, instead of ZFS we use 4 pairs of mirror disk to be able to stripping writing/reading between them.

    We’ve noticed we have better results with mirror disk (similar to RAID 10) than ZFS. Our SAN software is Nexenta community edition.

    Again, I want to thank the owner of this blog, we have learned a lot.

    Thanks!

  • Leave a Reply

    You must be logged in to post a comment.