On Fri, 2009-06-26 at 23:17 +0200, Andreas Dilger wrote: > On Jun 26, 2009 16:34 -0400, David Dillow wrote: > > On Fri, 2009-06-26 at 22:05 +0200, Andreas Dilger wrote: > > > In some cases there will of course be a need for multiple MGSes in > > > a single site (e.g. secure and open networks), which is fine as long > > > as clients don't try to mount from multiple MGSes at once. > > > > What kind of problems could this cause? > > > > We've had this configuration on both segments of Jaguar for some time > > now with no ill effects that we could attribute to mounting from two > > MGSes at the same time -- well, maybe that is not entirely true, as one > > is an MGS, and the other is a combined MGS/MDS. Some clients are talking > > with up to five separate combined MGS/MDS. > > Well, there is only ever a single MGC configured on a client at one time, > so if you have multiple MGSes I would suspect that this will cause the > clients to be evicted from all but the last MGS, and as a result any config > changes made to the first-mounted filesystems will not be seen by the > clients. This might not be noticable until you make a config change and > half of the clients don't notice e.g. the new OST or similar.
This may explain an issue we've seen about config changes not propagating, but that occurred on the last filesystem mounted, so I'm not sure. Are there any plans to eliminate this restriction? Data transfer nodes for gridftp want to be able to mount separate Lustre filesystems from different compute resources, which quite often have a MGS for each filesystem, especially if they are Cray systems that have seen upgrades from Lustre 1.4. Is it possible to split an MGT from a combined MGT/MDT, and then combine those into one MGT? That would be a migration path for those systems with multiple filesystems -- much better than a complete reformat, anyways. It would still force sites with mulitple systems that used to have internal MGS service to have all but one with an external MGS now. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
