There is nothing I can think of off the top of my head that would be a limiting factor preventing SSK from working with multiple file systems. Unfortunately my test environment is unavailable to look at this right now. If you can send some server logs I can take a look although I suspect Sebastien is more familiar with this code than I am now.
Jeremy On Wed, Jun 3, 2020 at 8:01 PM Hans Henrik Happe <[email protected]> wrote: > We only have one MGS with two filesystems. It evolved into to two due to > some reconfiguration and the end game is to remove one. Anyway, one of the > modifications to the new one was to have daily key refresh. Either that > made the refresh issue more likely or having two filesystems is not a good > idea. It seemed to work with two so we went on and started to rsync some > data over. Then we hit the refresh issue. For now I'm just asking if > multiple filesystems could cause issues. > > Anyway, I'm setting up a debug system to test if I can reproduce it with a > single fs. Then I'll get back with more info about the actual error. > > I've attached some client output from the failing system. This was on a > nodemap with a 120s expire key, for fast testing. It seems that there need > to be I/O during the refresh to hit this. > > A bit more info about the setup. All except mgs is configured for ski > messaging. mdt2ost is using the default nodemap. > > Cheers, > Hans Henrik > > > On 03.06.2020 18.01, Sebastien Buisson wrote: > > Hi, > > Do you use one shared MGS for all your file systems, or does each file system > have its own MGS? In the latter case, are the MGSes running on the same node? > > You are mentioning a key refresh issue, so I am wondering if you see this > issue with multiple file systems only, or if it occurs when you have just one > file system setup? > > Cheers, > Sebastien. > > > Le 3 juin 2020 à 15:07, Hans Henrik Happe <[email protected]> <[email protected]> a > écrit : > > Hi, > > I'm trying to hunt down an issue where SSK is failing key refresh on > 2.12.4. Mounting the filesystem works, but active sessions dies at refresh. > > First I would like to get a few things cleared. > > Is multiple Lustre filesystems on the same servers supported with SSK? > > If so, is it supported to use the same nodemap on each filesystem? > Obviously, with different keys for each fs. > > A mount from an ssh to the root account will create this keyring on > CentOS 7: > > # keyctl show > Session Keyring > 669565440 --alswrv 0 0 keyring: _ses > 458158660 --alswrv 0 65534 \_ keyring: _uid.0 > 129939379 --alswrv 0 0 \_ user: lustre:erda > > 65534 usually is nfsnobody but is does not exist on the system. Would > this be an issue? Even if nfsnobody existed? > > A mount through sudo will create this keyring on CentOS 7: > > # keyctl show > Session Keyring > 381836048 --alswrv 0 65534 keyring: _uid_ses.0 > 423400032 --alswrv 0 65534 \_ keyring: _uid.0 > 934942793 --alswrv 0 0 \_ user: lustre:erda > > Again is this a problem? > > > Cheers, > Hans Henrik > _______________________________________________ > lustre-discuss mailing > [email protected]http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > > _______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org >
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
