We have been using ZFS on our LFS for about the last 7 years. Back then, we were using ZFS 0.7 and lustre 2.10 and there was a significant decrease in metadata performance compared to ldiskfs. Most of our workflows at the time didn’t need a lot of metadata performance and so we were OK with the hit. In the meantime, we’ve had more small file I/O workflows come online and have been more focused on getting better metadata performance. About 2 years ago we upgraded the LFS to ZFS 2.0 and lustre 2.14 and saw a big improvement in ZFS metadata performance, although still not as good as ldiskfs or even straight NFS. Our setup still shows a bit faster MDT performance with ldiskfs but the gap is now ~10% rather than about a factor of 2 from the early life of this LFS. So bottom line, our experience shows that the difference between the two is a lot less than it used to be but ZFS still is slower than ldiskfs.
From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Jon Marshall via lustre-discuss <lustre-discuss@lists.lustre.org> Reply-To: Jon Marshall <jon.marsh...@cruk.cam.ac.uk> Date: Friday, July 21, 2023 at 5:35 AM To: "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: [EXTERNAL] [BULK] [lustre-discuss] ldiskfs patch rejected Rocky 8.6 CAUTION: This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the NASA SOC. Hi, Me again! I now have a successful build of lustre server with ZFS support, thanks all for your help with this. I'm now stuck on getting ldiskfs support to work - specifically it requires the kernel-debuginfo and debugsource packages, which on Rocky 8.6, don't exist for the kernel I'm building against (4.18.0-372.9.1.el8.x86_64). I've installed the only ones that exist, for kernel 4.18.0-477.15.1.el8_8 and hackily symlinked it to the current kernel to get lustre to build against it, which it does, but then it fails on applying patches, which I suppose is not surprising. I assumed that having the e2fsprogs devel installed would do the trick but the build is specific in looking in /usr/src/kernel and /usr/src/debug/kernel, neither of which have it. I'd like ldiskfs support as we currently run this fs on the MDT. However, we're only doing that because when we put the filesystem in place (which was a while ago) I'd read a fair amount warning against using ZFS on MDT due to potential performance issues. I've had a trawl through the mailing list and I can't see anyone having issues with newer ZFS versions so I guess my question is, is this still a thing? I've been planning to move to ZFS at some point anyway and it seems like this would be a good opportunity to do so but I thought I'd check and see if anyone had any advice. Thanks in advance Jon Jon Marshall High Performance Computing Specialist IT and Scientific Computing Team Cancer Research UK Cambridge Institute Li Ka Shing Centre | Robinson Way | Cambridge | CB2 0RE Web<http://www.cruk.cam.ac.uk/> | Facebook<http://www.facebook.com/cancerresearchuk> | Twitter<http://twitter.com/CR_UK> [Description: CRI Logo]<http://www.cruk.cam.ac.uk/>
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
