HowTo : Add Cache drives to a Zpool

The Cache drives (or L2ARC Cache) are used for frequently accessed data. In our system we have configured it with 320GB of L2ARC cache. This cache resides on MLC SSD drives which have significantly faster access times than traditional spinning media. This means that up to 320GB of the most frequently accessed data can be kept in an SSD cache, and when requested does not have to be read from spinning media. This greatly speeds up access time for files that are frequently used.

To add caching drives to your Zpool first run the format command to find the disks that you have in your system.

Format Command

Once you have run the format command, you should have a list of disks that you can select for your cache drives. Select the ones that you would like to use (Our cache drives are c4t2d0 and c4t3d0) and issue this command :

zpool add tank cache c4t2d0
zpool add tank cache c4t3d0

Once you issue that command, it should add the cache drives to the zpool. Run the “zpool status” command to make sure it was updated as expected.

ZFS Add Cache Drives
ZFS Cache Drives Added

Thursday, June 3rd, 2010 RAID, ZFS

6 Comments to HowTo : Add Cache drives to a Zpool

  • Do you know? if it “stripes” the cache to increase performance? So would it be faster with four 160G drives vs 2 320G? Assuming the rest of the specs are equal?

  • admin says:

    I don’t know if ZFS actually stripes the data across the cache drives or not. I would assume the read performance of cached blocked would be better with four 160GB drives than with two 320GB drives, but I have not tested to confirm. One thing to keep in mind when adding cache drives is that ZFS needs to use about 1-2GB of the ARC for each 100GB of cache drives. If you are planning to run a L2ARC of 600GB, then ZFS could use as much as 12GB of the ARC just to manage the cache drives. You will want to make sure your ZFS server has quite a bit more than 12GB of total RAM. I’d personally recommend using 24GB-48GB of system RAM if you are using 600GB of L2ARC.

  • wvandenberge says:

    The ZFS best practices guide mentions that the ZIL should be sized at X MB/sec * 10 sec. However I can not find any recomendation on sizing the L2ARC.

    I’m building a large (90TB) system, is there a rule of thumb that would give me optimum L2ARC size (assuming physical memory of 64GB)?

  • admin says:

    As a rule of thumb, expect each 100GB of L2ARC to use up 1-2GB of ARC. In theory, you should be able to use up to about 3TB of L2ARC on a 64GB server. But I would not recommend using all of your ARC that way. You will need some ARC for other things. For example, if you use deduping, you will need a lot of ARC available to handle that.

    You mentioned that you will have a 90TB system. The total raw disk space is not important when discussing caching. The working set size (the amount of information getting accessed frequently) is the really important value. What is your working set size? In most of our servers, we have a few hundred GB that gets accessed frequently regardless of how many TB of storage. If you can, try to get enough L2ARC to cache most of the working set.

  • […] SATA 6G ports might be useful for connecting one or more high speed SDDs for use as a ZFS cache drive […]

  • Leave a Reply

    You must be logged in to post a comment.