Hi, I figured out what was going on with this issue. Basically cache timeouts were causing a large number of uid numbers in an arbitrarily-timed directory listing to have expired cache records, which causes those records to be looked up again by the data provider (and thus blocking ‘ls -l’). To work around this issue now we currently setting the entry_cache_timeout to something arbitrarily high, i.e. 999999, I’m questioning whether or not this is the best approach. I’d like to use something like refresh_expired_interval, although based on my testing it appears that this does not update records for a trusted AD domain. I’ve also tried using enumeration, and that doesn’t seem to work either.
I suppose my question is this; is there a preferred method to keep cache records up-to-date for a trusted AD domain? Right now I am thinking about cron-tabbing an ‘ls -l’ of /home and allowing entry_cache_nowait_percentage to fill this function, although that seems hacky to me. Any advisement that could be provided would be greatly appreciated. Best, Dan Sullivan > On Jan 30, 2017, at 10:52 AM, Sullivan, Daniel [CRI] > <[email protected]> wrote: > > Hi, > > I have another question about sssd performance. I’m having a difficult time > doing a regularly performant ‘ls -l’ operation against /home, a mounted NFS > share of all of our users home directories. There are 667 entries in this > folder, and all of them have IDs that are resolvable via freeipa/sssd. We > are using an AD trusted domain. > > It is clear to me why an initial invocation of this lookup should take some > time (populating the local ldb cache). And it does. Usually around 5-10 > minutes, but sometimes longer. After the initial lookups are complete, the > output of ‘ls -l' renders fine, and I can inspect the local filesystem cache > using ldbsearch and see that it is populated. The issue is that if I wait a > while, or restart sssd, it appears that I have to go through all of these > lookups again to render the directory listing. > > I am trying to find an optimal configuration for sssd.conf that will allow a > performant ‘ls -l’ listing of a directory with a large number of different id > numbers assigned to filesystem objects to always return results immediately > from the local cache (after an initial invocation of this command for any > given directory). I think basically what I want is to have the ldb cache > always ‘up-to-date’, or at least have sssd willing to immediately dump what > it has without having to do a bunch of lookups while blocking the ‘ls -l’ > thread. If possible, whatever solution implemented should also survive a > restart of the sssd process. In short, aside from an initial invocation, I > never want ‘ls -l’ to take more than a few seconds. > > The issue described above is somewhat problematic because it appears to cause > contention on the sssd process effectively allowing a user doing ls -l /home > to inadvertently degrade system performance for another user. > > So far I have tried: > > 1) Implementing 'enumeration = true' for the [domain] section . This seems > to have no impact. It might be worthwhile to note that we are using an AD > trusted domain. > 2) Using the refresh_expired_interval configuration for the [domain] section > > I have read the following two documents in a decent level of detail: > > https://jhrozek.wordpress.com/2015/08/19/performance-tuning-sssd-for-large-ipa-ad-trust-deployments/ > https://jhrozek.wordpress.com/2015/03/11/anatomy-of-sssd-user-lookup/ > > It almost seems to me like the answer to this would be to keep the LDB cache > valid indefinitely (step 4 on > https://jhrozek.wordpress.com/2015/03/11/anatomy-of-sssd-user-lookup/). > > Presumably this is a problem that somebody has seen before. Would somebody > be able to advise on the best way to deal with this? I appreciate your help. > > Thank you, > > Dan > > -- > Manage your subscription for the Freeipa-users mailing list: > https://www.redhat.com/mailman/listinfo/freeipa-users > Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
