Re: [lustre-discuss] Lustre Client in a container
Thanks for all the answers. I was thinking of creating a new file system, starting from clean configuration, implementing quotas etc. For that I was looking for a way in which the systems can coexist, moving symbolic links while the folders are synchronized to the new system. In the process emptying disks of the old file system and moving them to the new one. This is a long process that might take more then a month, but can be done without disturbing normal cluster operation. As it doesn't seems to be possible in real life, I will have to reevaluate my options and come with a different migration schema. On Wed, Jan 3, 2018 at 1:49 PM, Patrick Farrell <p...@cray.com> wrote: > FWIW, as long as you don’t intend to use any interesting features (quotas, > etc), 1.8 clients were used with 2.5 servers at ORNL for some time with no > ill effects on the IO side of things. > > I’m not sure how much further that limited compatibility goes, though. > -- > *From:* Dilger, Andreas <andreas.dil...@intel.com> > *Sent:* Wednesday, January 3, 2018 4:20:56 AM > *To:* David Cohen > *Cc:* Patrick Farrell; lustre-discuss@lists.lustre.org > *Subject:* Re: [lustre-discuss] Lustre Client in a container > > On Dec 31, 2017, at 01:50, David Cohen <cda...@physics.technion.ac.il> > wrote: > > > > Patrick, > > Thanks for you response. > > I looking for a way to migrate from 1.8.9 system to 2.10.2, stable > enough to run the several weeks or more that it might take. > > Note that there is no longer direct support for upgrading from 1.8 to > 2.10. > > That said, are you upgrading the filesystem in place, or are you copying > the data from the 1.8.9 filesystem to the 2.10.2 filesystem? In the latter > case, the upgrade compatibility doesn't really matter. What you need is a > client that can mount both server versions at the same time. > > Unfortunately, no 2.x clients can mount the 1.8.x server filesystem > directly, so that does limit your options. There was a time of > interoperability with 1.8 clients being able to mount 2.1-ish servers, but > that doesn't really help you. You could upgrade the 1.8 servers to 2.1 or > later, and then mount both filesystems with a 2.5-ish client, or upgrade > the servers to 2.5. > > Cheers, Andreas > > > On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > > David, > > > > I have no direct experience trying this, but I would imagine not - > Lustre is a kernel module (actually a set of kernel modules), so unless the > container tech you're using allows loading multiple different versions of > *kernel modules*, this is likely impossible. My limited understanding of > container tech on Linux suggests that this would be impossible, containers > allow userspace separation but there is only one kernel/set of > modules/drivers. > > > > I don't know of any way to run multiple client versions on the same node. > > > > The other question is *why* do you want to run multiple client versions > on one node...? Clients are usually interoperable across a pretty generous > set of server versions. > > > > - Patrick > > > > > > From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on > behalf of David Cohen <cda...@physics.technion.ac.il> > > Sent: Saturday, December 30, 2017 11:45:15 AM > > To: lustre-discuss@lists.lustre.org > > Subject: [lustre-discuss] Lustre Client in a container > > > > Hi, > > Is it possible to run Lustre client in a container? > > The goal is to run two different client version on the same node, can it > be done? > > > > David > > > > > > ___ > > lustre-discuss mailing list > > lustre-discuss@lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > Cheers, Andreas > -- > Andreas Dilger > Lustre Principal Architect > Intel Corporation > > > > > > > > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
FWIW, as long as you don’t intend to use any interesting features (quotas, etc), 1.8 clients were used with 2.5 servers at ORNL for some time with no ill effects on the IO side of things. I’m not sure how much further that limited compatibility goes, though. From: Dilger, Andreas <andreas.dil...@intel.com> Sent: Wednesday, January 3, 2018 4:20:56 AM To: David Cohen Cc: Patrick Farrell; lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Lustre Client in a container On Dec 31, 2017, at 01:50, David Cohen <cda...@physics.technion.ac.il> wrote: > > Patrick, > Thanks for you response. > I looking for a way to migrate from 1.8.9 system to 2.10.2, stable enough to > run the several weeks or more that it might take. Note that there is no longer direct support for upgrading from 1.8 to 2.10. That said, are you upgrading the filesystem in place, or are you copying the data from the 1.8.9 filesystem to the 2.10.2 filesystem? In the latter case, the upgrade compatibility doesn't really matter. What you need is a client that can mount both server versions at the same time. Unfortunately, no 2.x clients can mount the 1.8.x server filesystem directly, so that does limit your options. There was a time of interoperability with 1.8 clients being able to mount 2.1-ish servers, but that doesn't really help you. You could upgrade the 1.8 servers to 2.1 or later, and then mount both filesystems with a 2.5-ish client, or upgrade the servers to 2.5. Cheers, Andreas > On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > David, > > I have no direct experience trying this, but I would imagine not - Lustre is > a kernel module (actually a set of kernel modules), so unless the container > tech you're using allows loading multiple different versions of *kernel > modules*, this is likely impossible. My limited understanding of container > tech on Linux suggests that this would be impossible, containers allow > userspace separation but there is only one kernel/set of modules/drivers. > > I don't know of any way to run multiple client versions on the same node. > > The other question is *why* do you want to run multiple client versions on > one node...? Clients are usually interoperable across a pretty generous set > of server versions. > > - Patrick > > > From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of > David Cohen <cda...@physics.technion.ac.il> > Sent: Saturday, December 30, 2017 11:45:15 AM > To: lustre-discuss@lists.lustre.org > Subject: [lustre-discuss] Lustre Client in a container > > Hi, > Is it possible to run Lustre client in a container? > The goal is to run two different client version on the same node, can it be > done? > > David > > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
On Dec 31, 2017, at 01:50, David Cohen <cda...@physics.technion.ac.il> wrote: > > Patrick, > Thanks for you response. > I looking for a way to migrate from 1.8.9 system to 2.10.2, stable enough to > run the several weeks or more that it might take. Note that there is no longer direct support for upgrading from 1.8 to 2.10. That said, are you upgrading the filesystem in place, or are you copying the data from the 1.8.9 filesystem to the 2.10.2 filesystem? In the latter case, the upgrade compatibility doesn't really matter. What you need is a client that can mount both server versions at the same time. Unfortunately, no 2.x clients can mount the 1.8.x server filesystem directly, so that does limit your options. There was a time of interoperability with 1.8 clients being able to mount 2.1-ish servers, but that doesn't really help you. You could upgrade the 1.8 servers to 2.1 or later, and then mount both filesystems with a 2.5-ish client, or upgrade the servers to 2.5. Cheers, Andreas > On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > David, > > I have no direct experience trying this, but I would imagine not - Lustre is > a kernel module (actually a set of kernel modules), so unless the container > tech you're using allows loading multiple different versions of *kernel > modules*, this is likely impossible. My limited understanding of container > tech on Linux suggests that this would be impossible, containers allow > userspace separation but there is only one kernel/set of modules/drivers. > > I don't know of any way to run multiple client versions on the same node. > > The other question is *why* do you want to run multiple client versions on > one node...? Clients are usually interoperable across a pretty generous set > of server versions. > > - Patrick > > > From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of > David Cohen <cda...@physics.technion.ac.il> > Sent: Saturday, December 30, 2017 11:45:15 AM > To: lustre-discuss@lists.lustre.org > Subject: [lustre-discuss] Lustre Client in a container > > Hi, > Is it possible to run Lustre client in a container? > The goal is to run two different client version on the same node, can it be > done? > > David > > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
On Mon, Jan 1, 2018 at 12:38 AM, Patrick Farrell <p...@cray.com> wrote: > > > Keeping in mind that the choice of NFS means that you don’t have the POSIX > guarantees provided by Lustre, so simultaneous access to the same files is > dicey unless it’s only reading. > That just means that during the migration phase the 2.10 filesystem should not be production yet or alternatively the 1.8 system is out of prodution and read-only which would anyhow be the goal of a migration or am I seeing this totally wrong? Good luck and thanks everyone on lustre-users for being an excelletn mailing list :) Eli > -- > *From:* lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on > behalf of Daniel Kobras <kob...@linux.de> > *Sent:* Sunday, December 31, 2017 12:26:49 PM > *To:* David Cohen > *Cc:* lustre-discuss@lists.lustre.org > *Subject:* Re: [lustre-discuss] Lustre Client in a container > > Hi David! > > Do you require both systems to be available as native Lustre filesystems > on all clients? Otherwise, reexporting one of the systems via NFS during > the migration phase will keep all data available but decouple the version > interdependence between servers and clients. In this situation, it’s > probably the least experimental option. > > Kind regards, > > Daniel > > > Am 31.12.2017 um 09:50 schrieb David Cohen < > cda...@physics.technion.ac.il>: > > > > Patrick, > > Thanks for you response. > > I looking for a way to migrate from 1.8.9 system to 2.10.2, stable > enough to run the several weeks or more that it might take. > > > > > > David > > > > On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > > David, > > > > I have no direct experience trying this, but I would imagine not - > Lustre is a kernel module (actually a set of kernel modules), so unless the > container tech you're using allows loading multiple different versions of > *kernel modules*, this is likely impossible. My limited understanding of > container tech on Linux suggests that this would be impossible, containers > allow userspace separation but there is only one kernel/set of > modules/drivers. > > > > I don't know of any way to run multiple client versions on the same node. > > > > The other question is *why* do you want to run multiple client versions > on one node...? Clients are usually interoperable across a pretty generous > set of server versions. > > > > - Patrick > > > > > > From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on > behalf of David Cohen <cda...@physics.technion.ac.il> > > Sent: Saturday, December 30, 2017 11:45:15 AM > > To: lustre-discuss@lists.lustre.org > > Subject: [lustre-discuss] Lustre Client in a container > > > > Hi, > > Is it possible to run Lustre client in a container? > > The goal is to run two different client version on the same node, can it > be done? > > > > David > > > > > > ___ > > lustre-discuss mailing list > > lustre-discuss@lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
Keeping in mind that the choice of NFS means that you don’t have the POSIX guarantees provided by Lustre, so simultaneous access to the same files is dicey unless it’s only reading. From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Daniel Kobras <kob...@linux.de> Sent: Sunday, December 31, 2017 12:26:49 PM To: David Cohen Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Lustre Client in a container Hi David! Do you require both systems to be available as native Lustre filesystems on all clients? Otherwise, reexporting one of the systems via NFS during the migration phase will keep all data available but decouple the version interdependence between servers and clients. In this situation, it’s probably the least experimental option. Kind regards, Daniel > Am 31.12.2017 um 09:50 schrieb David Cohen <cda...@physics.technion.ac.il>: > > Patrick, > Thanks for you response. > I looking for a way to migrate from 1.8.9 system to 2.10.2, stable enough to > run the several weeks or more that it might take. > > > David > > On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > David, > > I have no direct experience trying this, but I would imagine not - Lustre is > a kernel module (actually a set of kernel modules), so unless the container > tech you're using allows loading multiple different versions of *kernel > modules*, this is likely impossible. My limited understanding of container > tech on Linux suggests that this would be impossible, containers allow > userspace separation but there is only one kernel/set of modules/drivers. > > I don't know of any way to run multiple client versions on the same node. > > The other question is *why* do you want to run multiple client versions on > one node...? Clients are usually interoperable across a pretty generous set > of server versions. > > - Patrick > > > From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of > David Cohen <cda...@physics.technion.ac.il> > Sent: Saturday, December 30, 2017 11:45:15 AM > To: lustre-discuss@lists.lustre.org > Subject: [lustre-discuss] Lustre Client in a container > > Hi, > Is it possible to run Lustre client in a container? > The goal is to run two different client version on the same node, can it be > done? > > David > > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
Hi David! Do you require both systems to be available as native Lustre filesystems on all clients? Otherwise, reexporting one of the systems via NFS during the migration phase will keep all data available but decouple the version interdependence between servers and clients. In this situation, it’s probably the least experimental option. Kind regards, Daniel > Am 31.12.2017 um 09:50 schrieb David Cohen <cda...@physics.technion.ac.il>: > > Patrick, > Thanks for you response. > I looking for a way to migrate from 1.8.9 system to 2.10.2, stable enough to > run the several weeks or more that it might take. > > > David > > On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > David, > > I have no direct experience trying this, but I would imagine not - Lustre is > a kernel module (actually a set of kernel modules), so unless the container > tech you're using allows loading multiple different versions of *kernel > modules*, this is likely impossible. My limited understanding of container > tech on Linux suggests that this would be impossible, containers allow > userspace separation but there is only one kernel/set of modules/drivers. > > I don't know of any way to run multiple client versions on the same node. > > The other question is *why* do you want to run multiple client versions on > one node...? Clients are usually interoperable across a pretty generous set > of server versions. > > - Patrick > > > From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of > David Cohen <cda...@physics.technion.ac.il> > Sent: Saturday, December 30, 2017 11:45:15 AM > To: lustre-discuss@lists.lustre.org > Subject: [lustre-discuss] Lustre Client in a container > > Hi, > Is it possible to run Lustre client in a container? > The goal is to run two different client version on the same node, can it be > done? > > David > > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
Ah, yes, that will likely be a tricky one. You may simply have to bite the bullet, copy what you can early, and accept a downtime to finalize. Note also that there may be no kernel version for which you could compile both of those? Possibly some version of CentOS 6. From: David Cohen <cda...@physics.technion.ac.il> Sent: Sunday, December 31, 2017 2:50:05 AM To: Patrick Farrell Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Lustre Client in a container Patrick, Thanks for you response. I looking for a way to migrate from 1.8.9 system to 2.10.2, stable enough to run the several weeks or more that it might take. David On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> wrote: David, I have no direct experience trying this, but I would imagine not - Lustre is a kernel module (actually a set of kernel modules), so unless the container tech you're using allows loading multiple different versions of *kernel modules*, this is likely impossible. My limited understanding of container tech on Linux suggests that this would be impossible, containers allow userspace separation but there is only one kernel/set of modules/drivers. I don't know of any way to run multiple client versions on the same node. The other question is *why* do you want to run multiple client versions on one node...? Clients are usually interoperable across a pretty generous set of server versions. - Patrick From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org<mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of David Cohen <cda...@physics.technion.ac.il<mailto:cda...@physics.technion.ac.il>> Sent: Saturday, December 30, 2017 11:45:15 AM To: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: [lustre-discuss] Lustre Client in a container Hi, Is it possible to run Lustre client in a container? The goal is to run two different client version on the same node, can it be done? David ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre Client in a container
Patrick, Thanks for you response. I looking for a way to migrate from 1.8.9 system to 2.10.2, stable enough to run the several weeks or more that it might take. David On Sun, Dec 31, 2017 at 12:12 AM, Patrick Farrell <p...@cray.com> wrote: > David, > > > I have no direct experience trying this, but I would imagine not - Lustre > is a kernel module (actually a set of kernel modules), so unless the > container tech you're using allows loading multiple different versions of > *kernel modules*, this is likely impossible. My limited understanding of > container tech on Linux suggests that this would be impossible, containers > allow userspace separation but there is only one kernel/set of > modules/drivers. > > > I don't know of any way to run multiple client versions on the same node. > > > The other question is *why* do you want to run multiple client versions on > one node...? Clients are usually interoperable across a pretty generous > set of server versions. > > > - Patrick > > > > -- > *From:* lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on > behalf of David Cohen <cda...@physics.technion.ac.il> > *Sent:* Saturday, December 30, 2017 11:45:15 AM > *To:* lustre-discuss@lists.lustre.org > *Subject:* [lustre-discuss] Lustre Client in a container > > Hi, > Is it possible to run Lustre client in a container? > The goal is to run two different client version on the same node, can it > be done? > > David > > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] Lustre Client in a container
Hi, Is it possible to run Lustre client in a container? The goal is to run two different client version on the same node, can it be done? David ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
