Testing the L2ARC

We posted an article a while back that explained the cool L2ARC feature of ZFS.  We thought it would be fun to actually test the L2ARC and build a chart of the performance as a function of time.  To test and graph usefulness of L2ARC, we set up an iSCSI share on the ZFS server and then ran IOmeter from our test blade in our blade center.  We ran these tests over gigabit Ethernet.

IOmeter Test Details:
25GB working set
4k blocks
100% random
100% read
load 32 (constant)
four hour test

Every ten minutes during the test, we grabbed the “Last performance” values (IOPS, MB/sec) from IOmeter and wrote them down to build a performance chart.  Our goal was to be able to graph the performance as a function of time so we could illustrate the usefulness of the L2ARC.

We ran the same test using the Promise 610i (16 SATA drives in RAID10) box to get a comparison graph.  The Promise box is not a ZFS style solution and does not have any L2ARC style caching feature.  We expected the ZFS box to outperform the Promise box, and we expected the ZFS box to increase performance as a function of time because the L2ARC would become more populated the longer the test ran. 

The Promise box consistently delivered 2200 to 2300 IOPS every time we checked performance during the entire 4 hour test.  The ZFS box started by delivering 2532 IOPS at 10 minutes into the test and delivered 20873 IOPS by the end of the test.

Here is the graph of the performance results:

L2ARC Performance Testing

Initially, the two SAN boxes deliver similar performance, with the Promise box at 2200 IOPS and the ZFS box at 2500 IOPS.  However, the ZFS box with a L2ARC is able to magnify its performance by a factor of ten once the L2ARC is completely populated! 

Notice that ZFS limits how quickly the L2ARC is populated to reduce wear on the cache drives.  It takes a few hours to populate the L2ARC and achieve maximum performance.  That seems like a long time when running benchmarks, but it is actually a very short period of time in the life cycle of a typical SAN box.

Friday, July 30th, 2010 ZFS

2 Comments to Testing the L2ARC

  • your data set is smaller than your l2arc, I would be more interested in seeing a benchmark where l2arc is 1/10 or more likely 1/20 the total data set, which would be more similar to a 20tb array with 10tb used and four 250gb SSD drives

  • admin says:

    screamingservers: Feel free to run some benchmarks of your own. There are plenty of possible scenarios to test. For the L2ARC testing, we intentionally chose a working set that was bigger than the ARC but smaller than the L2ARC.

  • Leave a Reply

    You must be logged in to post a comment.