The uid cache is purged after each gids map update, so each update of the gids map should take about the same amount of time. I purged the uid cache because I wanted to make sure I caught any updates to uids as well. This cache is most effective when you have users in several groups, so you don't need repeated calls to getpwnam() for the same uid. And I think it explains why the times now are substantially shorter, even compared to your C test which (I'm assuming) didn't cache the uids.
I apologize for the problems you're having with the Gna! spam filtering. I find it ironic that mail coming from a company largely responsible for connecting the 'net is being blocked. -Chris On Thu, 12/21/06 08:25a EST, Jeff Squyres wrote: > > On Dec 20, 2006, at 9:51 PM, Chris Dunlap wrote: > > > In the getgrent() loop in _gids_hash_create(), I replaced the call > > to getpwnam() with _gids_user_to_uid() which caches the uid lookup in > > a second hash. I was just curious to know how much of an effect the > > uid caching had on very large passwd/group files, especially compared > > to the 27min it took 0.5.6 based on the initial log. > > Ah! I missed that. > > I put some printf's around the gid hash map loop and ran munged in > the foreground. > > 1. With skeletal passwd files (i.e., correct NIS configuration): > > 1a. the first loop took 12 seconds (with multiple runs, there was a > pretty wide variance on this -- I saw times as high as 31 seconds, so > I can only assume that this time is related to other factors, such as > the present load on the NIS server). > 1b. I then did "touch /etc/group" to force munged to reprocess the file. > 1c. the next loop took about 12 seconds. > 1d. I repeated the "touch /etc/group" step multiple times to see if > the times would vary; they were consistently in the 11-13 second range. > > 2. With full passwd files (i.e., incorrect NIS configuration, so I'm > not sure if this data point really matters): > > 2a. The first loop took 256 seconds > 2b. As above, I executed "touch /etc/group" to for munged to > reprocess the file. > 2c. Subsequent times through the loop took ~250-260 seconds. > > I'm at a loss as to why the time is *significantly* shorter than it > used to be, even for the *first* iteration through the loop. My own > C test still takes ~600 seconds to getrent() across all entries. > > One issue could be time of day when I executed these tests: the tests > in this mail were conducted between 7-8am US eastern time (4-5am US > pacific time, where the NIS server is located). Tests in my prior e- > mail were run about 12 hours difference from that (I don't remember > the exact times, but it was late afternoon / early evening US eastern > time), and loads on the NIS server could have been much different...? > > Regardless, the processing in munged itself got much faster, even if > there's little difference between the first loop and subsequent loops. > > (I'm quite sure that this post won't reach the list; Gna! seems > determined to classify my mail as junk :-( ) _______________________________________________ munge-users mailing list [email protected] https://mail.gna.org/listinfo/munge-users
