Re: Seeking clarification for nscd invalidation
[is the glibc maintainers mailing list still the correct venue?] Hi, I really appreciate your help, it is insightful. On Wed, Jul 06, 2022 at 04:57:47PM -0400, Carlos O'Donell wrote: > On Wed, Jul 6, 2022 at 10:10 AM Marc Haber > wrote: > > On Wed, Jul 06, 2022 at 09:29:22AM -0400, Carlos O'Donell wrote: > > > You can verify that nscd is catching the cases you care about by running > > > it in '--debug' mode to let you see the cache invalidation. > > > > I regret to say that I don't use nscd in any of my installations. My > > last superficial contact to nscd was like two decades ago. > > > > Would it help with an educated guess to install nscd, let it run in > > --debug mode, run our test suite and compare nscd's output with the > > output it generates running our test suite without the explicit > > invalidation? > > That sounds like a good approach. You should be able to compare > the logs of both runs looking for the missing explicit invalidation > and you should see that nscd detects the database changes. I have uploaded two typescripts from nscd -d with debug_level 0 to https://q.bofh.de/~mh/stuff/ - one with the invalidation code in adduser in place, one with the invalidation code removed. Both logs show the adduser test suite running. Unfortunately, that doesnt look like I can make any sense from that. I see nscd seeing changes to /etc/passwd and /etc/group, so I guess that what we do is correctly reflected by nscd even without the explicit invalidation. Additionally, I guess that adduser is invalidating caches that don't need invalidation (logs showing explicit invalidations without the software noticing changed files). I am wondering about the following though: monitored file `/etc/group` was deleted, removing watch monitored file `/etc/group` was created, adding watch without seeing actual caches being invalidated. Could it be possible that /etc/group is rewritten and renamed and that nscd is only watching for changes? Or am I missing something here? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Re: Seeking clarification for nscd invalidation
On Wed, Jul 6, 2022 at 10:10 AM Marc Haber wrote: > On Wed, Jul 06, 2022 at 09:29:22AM -0400, Carlos O'Donell wrote: > > You can verify that nscd is catching the cases you care about by running > > it in '--debug' mode to let you see the cache invalidation. > > I regret to say that I don't use nscd in any of my installations. My > last superficial contact to nscd was like two decades ago. > > Would it help with an educated guess to install nscd, let it run in > --debug mode, run our test suite and compare nscd's output with the > output it generates running our test suite without the explicit > invalidation? That sounds like a good approach. You should be able to compare the logs of both runs looking for the missing explicit invalidation and you should see that nscd detects the database changes. You may need to increase /etc/nscd.conf debug-level to 2. Cheers, Carlos.
Re: Seeking clarification for nscd invalidation
Hi Carlos, thank you for commenting. I am one of the adduser maintainers in Debian and we would like to ditch the parts of our code that invalidate nscd explicitly in one of the future versions of the software. On Wed, Jul 06, 2022 at 09:29:22AM -0400, Carlos O'Donell wrote: > You can verify that nscd is catching the cases you care about by running > it in '--debug' mode to let you see the cache invalidation. I regret to say that I don't use nscd in any of my installations. My last superficial contact to nscd was like two decades ago. Would it help with an educated guess to install nscd, let it run in --debug mode, run our test suite and compare nscd's output with the output it generates running our test suite without the explicit invalidation? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Re: Seeking clarification for nscd invalidation
On Mon, Jul 4, 2022 at 3:30 PM Marc Haber wrote: > adduser still has code to invalidate nscd cache after doing changes to > the user database. I would like to get rid of this and just document > that people using nscd should use the provided hook to invalidate their > nscd cache after creating or deleting users. > > I am wondering whether this is actually needed any more. The nscd source > code contains numerous calls to inotify, and > https://man7.org/linux/man-pages/man8/nscd.8.html suggests that it > actually notices changes to the passwd/group "database" files and > invalidates automatically at least since 2015. The nscd manpage in > Debian seems to have been taken from a different source. I guess it was > not updated. > > Do we still need to invalidate nscd cache "manually" or will nscd cover > the basic use cases automatically? I'm a glibc maintainer. In 2015 nscd's inotify support was improved to cover many more cases of editors changing the files with specific patterns of file system accesses. You can verify that nscd is catching the cases you care about by running it in '--debug' mode to let you see the cache invalidation. If cases of filesystem changes are not covered then nscd should be fixed. nscd is still being used by Guix and other distributions. Fedora is no longer going to ship nscd in favour of sssd as a caching framework. The future of nscd is likely going to be to simplify the code and provide a very thin local cache, but this is still up for discussion. Cheers, Carlos.
