Re: [lustre-discuss] Cannot mount MDT after upgrading from Lustre 2.12.6 to 2.15.3
On Oct 1, 2023, at 00:36, Tung-Han Hsieh via lustre-discuss wrote: > I should apologize for replying late. Here I would like to clarify why in my > opinion the Lustre ldiskfs code is not self-contained. > > In the past, to compile lustre with ldiskfs, we needed to patch Linux kernel > using the patches provided by Lustre source code. And YES, for the recent > Lustre versions, the necessary patches are fewer, and even OK without > applying any patches to Linux kernel. However, there are another patches to > Lustre ldiskfs code, namely copying Linux kernel ext4fs code into Lustre > source tree, applying patches provided by Lustre, and make it becomes the > ldiskfs code, during the compile time. Hello T.H., it is true that Lustre needs to patch the ldiskfs code in order to properly integrate with the ext4/jbd2 transaction handling, and to add some features that ext4 is lacking that Lustre depends on. Ideally we could get these features integrated into the upstream ext4 code to avoid the need to patch it, or at least minimize the number of patches, but that is often difficult and time consuming and doesn't get done as often as anyone would want it to. Lustre is an open-source project, and if you are depending on the Linux-5.4 stable kernel branch for your servers, it would be welcome for you to submit patches to update the kernel patch series if there are issues arising with these patches with a new stable kernel, as other developers maintain the kernel patch series for the distros that they are interested in (primarily RHEL derivatives, but more recently Ubuntu). > Here is the compilation log indicating the patching procedure. To compile > lustre-2.15.3 with Linux-5.4.135, after running > > ./configure --prefix=/opt/lustre --with-linux=/usr/src/linux-5.4.135 > --with-o2ib=no --with-ldiskfsprogs=/opt/e2fs --enable-mpitests=no > > and then run "make". The log reads: > > > rm -rf linux-stage linux sources trace > mkdir -p linux-stage/fs/ext4 linux-stage/include/linux \ > linux-stage/include/trace/events > cp /usr/src/linux-5.4.135/fs/ext4/file.c > /usr/src/linux-5.4.135/fs/ext4/ioctl.c /usr/src/linux-5.4.135/fs/ext4/dir.c > linux-stage/fs/ext4 > if test -n "" ; then \ > cp linux-stage/include/linux; \ > fi > if test -n "/usr/src/linux-5.4.135/include/trace/events/ext4.h" ; then \ > cp /usr/src/linux-5.4.135/include/trace/events/ext4.h > linux-stage/include/trace/events; \ > fi > ln -s ../../ldiskfs/kernel_patches/patches linux-stage/patches > ln -s ../../ldiskfs/kernel_patches/series/ldiskfs-5.4.136-ml.series > linux-stage/ > series > cd linux-stage && quilt push -a -q > Applying patch patches/rhel8/ext4-inode-version.patch > Applying patch patches/linux-5.4/ext4-lookup-dotdot.patch > Applying patch patches/suse15/ext4-print-inum-in-htree-warning.patch > Applying patch patches/rhel8/ext4-prealloc.patch > Applying patch patches/ubuntu18/ext4-osd-iop-common.patch > Applying patch patches/ubuntu19/ext4-misc.patch > Applying patch patches/rhel8/ext4-mballoc-extra-checks.patch > Applying patch patches/linux-5.4/ext4-hash-indexed-dir-dotdot-update.patch > Applying patch patches/linux-5.4/ext4-kill-dx-root.patch > Applying patch patches/rhel7.6/ext4-mballoc-pa-free-mismatch.patch > Applying patch patches/linux-5.4/ext4-data-in-dirent.patch > Applying patch patches/rhel8/ext4-nocmtime.patch > Applying patch patches/base/ext4-htree-lock.patch > Applying patch patches/linux-5.4/ext4-pdirop.patch > Applying patch patches/rhel8/ext4-max-dir-size.patch > Applying patch > patches/rhel8/ext4-corrupted-inode-block-bitmaps-handling-patches.patch > Applying patch > patches/linux-5.4/ext4-give-warning-with-dir-htree-growing.patch > Applying patch patches/ubuntu18/ext4-jcb-optimization.patch > Applying patch patches/linux-5.4/ext4-attach-jinode-in-writepages.patch > Applying patch patches/rhel8/ext4-dont-check-before-replay.patch > Applying patch > patches/rhel7.6/ext4-use-GFP_NOFS-in-ext4_inode_attach_jinode.patch > Applying patch patches/rhel7.6/ext4-export-orphan-add.patch > Applying patch patches/rhel8/ext4-export-mb-stream-allocator-variables.patch > Applying patch patches/ubuntu19/ext4-iget-with-flags.patch > Applying patch patches/linux-5.4/export-ext4fs-dirhash-helper.patch > Applying patch patches/linux-5.4/ext4-misc.patch > Applying patch patches/linux-5.4/ext4-simple-blockalloc.patch > Applying patch patches/linux-5.4/ext4-xattr-disable-credits-check.patch > Applying patch patches/base/ext4-no-max-dir-size-limit-for-iam-objects.patch > Applying patch patches/rhel8/ext4-ialloc-uid-gid-and-pass-owner-down.patch > Applying patch patches/base/ext4-projid-xattrs.patch > Applying patch patches/linux-5.4/ext4-enc-flag.patch > Applying patch patches/base/ext4-delayed-iput.patch > Now at patch patches/base/ext4-delayed-iput.patch >
Re: [lustre-discuss] Adding lustre clients into the Debian
On Oct 1, 2023, at 05:54, Arman Khalatyan via lustre-discuss wrote: > > Hello everyone, > > We are in the process of integrating the Lustre client into Debian. Are there > any legal concerns or significant obstacles to this? We're curious why it > hasn't been included in the official Debian repository so far. There used to > be an old, unmaintained Lustre client branch dating back to around 2013. I don't think there is any particular barrier to add a Debian Lustre package today. As you wrote, there was some effort put into that in the past, but I think there wasn't someone with the time and Debian-specific experience maintain it and get it included into a release. Lustre is GPLv2 licensed, with some LGPL user library API code, so there aren't any legal issues. The Lustre CI system builds Ubuntu clients, and I'd be happy to see patches to improve the Debian packaging in the Lustre tree. Most Lustre users are using RHEL derivatives, and secondarily Ubuntu, so there hasn't been anyone to work on specific Debian packaging in some time. Thomas (CC'd) was most active on the Debian front, and might be able to comment on this in more detail, and hopefully can also help review the patches. > You can check our wishlist on Debian here: https://bugs.debian.org/1053214 > > At AIP, one of our colleagues is responsible for maintaining Astropy, so we > have some experience with Debian. > > I've also set up a CI system in our GitLab, which includes a simple build and > push to a public S3 bucket. This is primarily for testing purposes to see if > it functions correctly for others... There is an automated build farm for Lustre, with patches submitted via Gerrit (https://wiki.lustre.org/Using_Gerrit), and while there aren't Debian build/test nodes to test the correctness of changes for Debian, this will at least avoid regressions with Ubuntu or, though unlikely, with RHEL. My assumption would be that any Debian-specific changes would continue to work with Ubuntu? Depending on how actively you want to build/test Lustre, it is also possible to configure your CI system to asynchronously follow patches under development in Gerrit to provide build and/or test feedback on the patches before they land. The contrib/scripts/gerrit_checkpatch.py script follows new patch submissions and runs code style reviews on patches after they are pushed, and posts comments back to Gerrit. A step beyond this, once your CI system is working reliably, it is possible to also post review comments directly into the patches in Gerrit, as the Gerrit Janitor does (https://github.com/verygreen/lustretester/). The gerrit_build-and-test-new.py script is derived from gerrit_checkpatch.py, but implements a more complex set of operations on each patch - static code analysis, build, test. It will add both general patch comments on success/failure of the build/test, or comments on specific lines in the patch. In some cases, Gerrit Janitor will also give negative code reviews in cases when a newly-added regression test added by a patch is failing regularly in its testing. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Adding lustre clients into the Debian
On Oct 1, 2023, at 05:54, Arman Khalatyan via lustre-discuss mailto:lustre-discuss@lists.lustre.org>> wrote: Hello everyone, We are in the process of integrating the Lustre client into Debian. Are there any legal concerns or significant obstacles to this? We're curious why it hasn't been included in the official Debian repository so far. There used to be an old, unmaintained Lustre client branch dating back to around 2013. I don't think there is any particular barrier to add a Debian Lustre package today. As you wrote, there was some effort put into that in the past, but I think there wasn't someone with the time and Debian-specific experience maintain it and get it included into a release. Lustre is GPLv2 licensed, with some LGPL user library API code, so there aren't any legal issues. The Lustre CI system builds Ubuntu clients, and I'd be happy to see patches to improve the Debian packaging in the Lustre tree. Most Lustre users are using RHEL derivatives, and secondarily Ubuntu, so there hasn't been anyone to work on specific Debian packaging in some time. Thomas (CC'd) was most active on the Debian front, and might be able to comment on this in more detail, and hopefully can also help review the patches. You can check our wishlist on Debian here: https://bugs.debian.org/1053214 At AIP, one of our colleagues is responsible for maintaining Astropy, so we have some experience with Debian. I've also set up a CI system in our GitLab, which includes a simple build and push to a public S3 bucket. This is primarily for testing purposes to see if it functions correctly for others... There is an automated build farm for Lustre, with patches submitted via Gerrit (https://wiki.lustre.org/Using_Gerrit), and while there aren't Debian build/test nodes to test the correctness of changes for Debian, this will at least avoid regressions with Ubuntu or, though unlikely, with RHEL. My assumption would be that any Debian-specific changes would continue to work with Ubuntu? Depending on how actively you want to build/test Lustre, it is also possible to configure your CI system to asynchronously follow patches under development in Gerrit to provide build and/or test feedback on the patches before they land. The contrib/scripts/gerrit_checkpatch.py script follows new patch submissions and runs code style reviews on patches after they are pushed, and posts comments back to Gerrit. A step beyond this, once your CI system is working reliably, it is possible to also post review comments directly into the patches in Gerrit, as the Gerrit Janitor does (https://github.com/verygreen/lustretester/). The gerrit_build-and-test-new.py script is derived from gerrit_checkpatch.py, but implements a more complex set of operations on each patch - static code analysis, build, test. It will add both general patch comments on success/failure of the build/test, or comments on specific lines in the patch. In some cases, Gerrit Janitor will also give negative code reviews in cases when a newly-added regression test added by a patch is failing regularly in its testing. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] Adding lustre clients into the Debian
Hello everyone, We are in the process of integrating the Lustre client into Debian. Are there any legal concerns or significant obstacles to this? We're curious why it hasn't been included in the official Debian repository so far. There used to be an old, unmaintained Lustre client branch dating back to around 2013. You can check our wishlist on Debian here: https://bugs.debian.org/1053214 At AIP, one of our colleagues is responsible for maintaining Astropy, so we have some experience with Debian. I've also set up a CI system in our GitLab, which includes a simple build and push to a public S3 bucket. This is primarily for testing purposes to see if it functions correctly for others... Greetings from Potsdam, Arman -- ** * Dr. Arman Khalatyan IT-eScience -SuperComputing * * Phone : +49-331-7499-528 * * Email : akhalat...@aip.deFAX : +49-331-7499-309 * ** ** * Leibniz-Institut für Astrophysik Potsdam (AIP) * * An der Sternwarte 16, 14482 Potsdam, Germany * ** * Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach * * Stiftung bürgerlichen Rechts * * Stiftungsverzeichnis Brandenburg: 26 742-00/7026 * ** ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Cannot mount MDT after upgrading from Lustre 2.12.6 to 2.15.3
Dear All, I should apologize for replying late. Here I would like to clarify why in my opinion the Lustre ldiskfs code is not self-contained. In the past, to compile lustre with ldiskfs, we needed to patch Linux kernel using the patches provided by Lustre source code. And YES, for the recent Lustre versions, the necessary patches are fewer, and even OK without applying any patches to Linux kernel. However, there are another patches to Lustre ldiskfs code, namely copying Linux kernel ext4fs code into Lustre source tree, applying patches provided by Lustre, and make it becomes the ldiskfs code, during the compile time. Here is the compilation log indicating the patching procedure. To compile lustre-2.15.3 with Linux-5.4.135, after running ./configure --prefix=/opt/lustre --with-linux=/usr/src/linux-5.4.135 --with-o2ib=no --with-ldiskfsprogs=/opt/e2fs --enable-mpitests=no and then run "make". The log reads: rm -rf linux-stage linux sources trace mkdir -p linux-stage/fs/ext4 linux-stage/include/linux \ linux-stage/include/trace/events cp /usr/src/linux-5.4.135/fs/ext4/file.c /usr/src/linux-5.4.135/fs/ext4/ioctl.c /usr/src/linux-5.4.135/fs/ext4/dir.c linux-stage/fs/ext4 if test -n "" ; then \ cp linux-stage/include/linux; \ fi if test -n "/usr/src/linux-5.4.135/include/trace/events/ext4.h" ; then \ cp /usr/src/linux-5.4.135/include/trace/events/ext4.h linux-stage/include/trace/events; \ fi ln -s ../../ldiskfs/kernel_patches/patches linux-stage/patches ln -s ../../ldiskfs/kernel_patches/series/ldiskfs-5.4.136-ml.series linux-stage/ series cd linux-stage && quilt push -a -q Applying patch patches/rhel8/ext4-inode-version.patch Applying patch patches/linux-5.4/ext4-lookup-dotdot.patch Applying patch patches/suse15/ext4-print-inum-in-htree-warning.patch Applying patch patches/rhel8/ext4-prealloc.patch Applying patch patches/ubuntu18/ext4-osd-iop-common.patch Applying patch patches/ubuntu19/ext4-misc.patch Applying patch patches/rhel8/ext4-mballoc-extra-checks.patch Applying patch patches/linux-5.4/ext4-hash-indexed-dir-dotdot-update.patch Applying patch patches/linux-5.4/ext4-kill-dx-root.patch Applying patch patches/rhel7.6/ext4-mballoc-pa-free-mismatch.patch Applying patch patches/linux-5.4/ext4-data-in-dirent.patch Applying patch patches/rhel8/ext4-nocmtime.patch Applying patch patches/base/ext4-htree-lock.patch Applying patch patches/linux-5.4/ext4-pdirop.patch Applying patch patches/rhel8/ext4-max-dir-size.patch Applying patch patches/rhel8/ext4-corrupted-inode-block-bitmaps-handling-patches.patch Applying patch patches/linux-5.4/ext4-give-warning-with-dir-htree-growing.patch Applying patch patches/ubuntu18/ext4-jcb-optimization.patch Applying patch patches/linux-5.4/ext4-attach-jinode-in-writepages.patch Applying patch patches/rhel8/ext4-dont-check-before-replay.patch Applying patch patches/rhel7.6/ext4-use-GFP_NOFS-in-ext4_inode_attach_jinode.patch Applying patch patches/rhel7.6/ext4-export-orphan-add.patch Applying patch patches/rhel8/ext4-export-mb-stream-allocator-variables.patch Applying patch patches/ubuntu19/ext4-iget-with-flags.patch Applying patch patches/linux-5.4/export-ext4fs-dirhash-helper.patch Applying patch patches/linux-5.4/ext4-misc.patch Applying patch patches/linux-5.4/ext4-simple-blockalloc.patch Applying patch patches/linux-5.4/ext4-xattr-disable-credits-check.patch Applying patch patches/base/ext4-no-max-dir-size-limit-for-iam-objects.patch Applying patch patches/rhel8/ext4-ialloc-uid-gid-and-pass-owner-down.patch Applying patch patches/base/ext4-projid-xattrs.patch Applying patch patches/linux-5.4/ext4-enc-flag.patch Applying patch patches/base/ext4-delayed-iput.patch Now at patch patches/base/ext4-delayed-iput.patch If you check the Lustre source code, before running "make", the directory "Lustre-2.15.3/ldiskfs/linux-stage" does not exist. After running "make", it was created by the above patching procedure. Look into the Lustre codes further, there is a directory providing the patches to create ldiskfs code: lustre-2.15.3/ldiskfs/kernel_patches/series Obviously these patches highly depend on the version of Linux kernel. We have to balance the requirement to drive our hardware, the long-term maintained Linux kernel (see https://www.kernel.org/), and the compatibility with Lustre. For now, we are lucky to find vanilla Linux-5.4.135 fulfilling the above requirements. In case that Linux-5.4.135 has bugs in driving our fiber storage device, and we have to change to, say, Linux-5.4.150 or later, then we will get into trouble. This rare situation did happen before. So it would be very appreciated if, in the future release, the ldiskfs code could be a self-contained code similar to ZFS, instead of copying from Linux kernel and applying patches to
