On Sat, Mar 2, 2024 at 6:13 AM Konstantin Belousov <k...@freebsd.org> wrote: > > CAUTION: This email originated from outside of the University of Guelph. Do > not click links or open attachments unless you recognize the sender and know > the content is safe. If in doubt, forward suspicious emails to > ith...@uoguelph.ca. > > > On Sat, Mar 02, 2024 at 05:40:08AM -0800, Rick Macklem wrote: > > On Fri, Mar 1, 2024 at 10:51 PM Konstantin Belousov <k...@freebsd.org> > > wrote: > > > > > > CAUTION: This email originated from outside of the University of Guelph. > > > Do not click links or open attachments unless you recognize the sender > > > and know the content is safe. If in doubt, forward suspicious emails to > > > ith...@uoguelph.ca. > > > > > > > > > On Fri, Mar 01, 2024 at 06:23:56AM -0800, Rick Macklem wrote: > > > > On Fri, Mar 1, 2024 at 12:00 AM Ronald Klop <ronald-li...@klop.ws> > > > > wrote: > > > > > > > > > > Interesting read. > > > > > > > > > > Would it be possible to separate locking for admin actions like a > > > > > client mounting an fs from traffic flowing for file operations? > > > > Well, the NFS server does not really have any concept of a mount. > > > > What I am referring to is the ClientID maintained for NFSv4 mounts, > > > > which all the open/lock/session/layout state hangs off of. > > > > > > > > For most cases, this state information can safely be accessed/modified > > > > via a mutex, but there are three exceptions: > > > > - creating a new ClientID (which is done by the ExchangeID operation) > > > > and typically happens when a NFS client does a mount. > > > > - delegation Recall (which only happens when delegations are enabled) > > > > One of the reasons delegations are not enabled by default on the > > > > FreeBSD server. > > > > - the DestroyClientID which is typically done by a NFS client during > > > > dismount. > > > > For these cases, it is just too difficult to do them without sleeping. > > > > As such, there is a sleep lock which the nfsd threads normally acquire > > > > shared > > > > when doing NFSv4 operations, but for the above cases the lock is aquired > > > > exclusive. > > > > - I had to give the exclusive lock priority over shared lock > > > > acquisition (it is a > > > > custom locking mechanism with assorted weirdnesses) because without > > > > that someone reported that new mounts took up to 1/2hr to occur. > > > > (The exclusive locker waited for 30min before all the other nfsd > > > > threads > > > > were not busy.) > > > > Because of this priority, once a nfsd thread requests the exclusive > > > > lock, > > > > all other nfsd threads executing NFSv4 RPCs block after releasing > > > > their > > > > shared lock, until the exclusive locker releases the exclusive lock. > > > Normal lockmgr locks + TDP_DEADLKTREAT private thread flag provide the > > > property of pref. exclusive waiters in presence of the shared waiters. > > > I think this is what you described above. > > It also has some weird properties, like if there are multiple requestors > > for the exclusive lock, once one thread gets it (the threads are nfsd worker > > threads and indistinct), the others that requested an exclusive thread are > > unblocked without the lock being issued to them. > This sounds to me as LK_SLEEPFAIL feature of lockmgr. > Do not underestimate the amount of weird features in it. Yep, sounds like it. I should take a look to see if lockmgr will work instead of the "rolled my own".
I should also take another look at new client creation, to see if there is a way to do it that doesn't require the exclusive lock (a lot of that code is 20years old now). rick > > > They then check if the exclusive lock is still needed (usually not, since > > the other thread has dealt with the case where it was needed) and > > then they can acquire a shared lock. > > Without this, there were cases where several threads would acquire > > the exclusive lock and then discover that the lock was not needed and > > just release it again. > > > > It also uses an assortment of weird flags/call args. > > > > rick > >
