Hello, I found that some buffers that could be L2ARC eligible are not flagged such, leading to some performance impact.
As a test I ran the same IO workload 10 times in a raw. It is a metadata only workload (files listing). l2arc_noprefetch=0. Here is the activity of my pool disks during this test : http://img15.hostingpics.net/pics/958365941.png We can see that the first files' listing takes about 3 hours, during which the SSD cache is fed. The other runs take about 35 minutes each, read IOs are made from SSD cache but also from HDD disks, cache is no more fed. After some debugging, I found that the buffers which are not L2-backed are metadata buffers, as the ones which are in the cache, but are not ARC_FLAG_L2CACHE flagged. I then quickly modified ZFS code to add ARC_FLAG_L2CACHE to each header, and here is the result : http://img15.hostingpics.net/pics/538922792.png All IOs now come from the cache, leading to a nice performance improvement. Perfect. So I must now understand why some buffers are not ARC_FLAG_L2CACHE flagged. Going deeper into the code, ARC_FLAG_L2CACHE is set by arc_read(), depending on its arc_flags parameter. arc_read() is called 22 times in ZFS code, only 2 calls are made with arc_flags containing ARC_FLAG_L2CACHE. I modified dbuf.c so that each one of its 4 arc_read() calls use ARC_FLAG_L2CACHE, it improved the situation. My question is then, can't we make all the arc_read() calls use ARC_FLAG_L2CACHE, when the pool has a L2 cache ? Thank you very much, Ben ------------------------------------------- openzfs-developer Archives: https://www.listbox.com/member/archive/274414/=now RSS Feed: https://www.listbox.com/member/archive/rss/274414/28015062-cce53afa Modify Your Subscription: https://www.listbox.com/member/?member_id=28015062&id_secret=28015062-f966d51c Powered by Listbox: http://www.listbox.com
