On Mon, 2012-01-02 at 11:54 -0900, Erinn Looney-Triggs wrote: > On 01/02/2012 11:40 AM, Jakub Hrozek wrote: > > On Mon, Jan 02, 2012 at 12:53:29PM -0500, Simo Sorce wrote: > >> On Mon, 2012-01-02 at 17:29 +0100, Jakub Hrozek wrote: > >>> On Mon, Jan 02, 2012 at 10:00:02AM -0500, Simo Sorce wrote: > >>>> On Sat, 2011-12-31 at 01:35 -0900, Erinn Looney-Triggs wrote: > >>>>> On 12/30/2011 07:19 PM, JR Aquino wrote: > >>>>>> > >>>>>> On Dec 30, 2011, at 5:45 PM, Erinn Looney-Triggs wrote: > >>>>>> > >>>>>>> I have been slowly rolling out FreeIPA to my systems, trying to track > >>>>>>> differences/changes. One of the most noticeable has been a large slow > >>>>>>> down in file access times. > >>>>>>> > >>>>>>> Let me explain as best as I can. I use AIDE to track the file system > >>>>>>> (think tripwire) and it runs checks once a day. During these checks it > >>>>>>> is scanning (almost) the entire file system and comparing it to a > >>>>>>> stored > >>>>>>> database. On a moderately powered system with ~151k files, an AIDE run > >>>>>>> will usually take ~30 minutes. After the system becomes an IPA client > >>>>>>> the same run will generally take ~90-120 minutes. Un-install the > >>>>>>> ipa-client, back to ~30 minutes for an AIDE run. > >>>>>>> > >>>>>>> Now clearly a lot of lookups are being done for user names and group > >>>>>>> names, and this will have a performance hit that is dependant on the > >>>>>>> network. However, the odd thing is that even when running on the IPA > >>>>>>> server itself the slowdown is still the same. > >>>>>>> > >>>>>>> Not sure if this is an IPA problem, an SSSD problem, a bit of both, or > >>>>>>> neither, perhaps it is just the way it is, but a slowdown of 3-4x > >>>>>>> seems > >>>>>>> a bit much to me. Clearly the results are not scientific, however, > >>>>>>> they > >>>>>>> have been generally reproducible since I started rolling IPA out. > >>>>>>> > >>>>>>> As a side note this slowdown has also broken bacula backups, as the > >>>>>>> bacula client is scanning the filesystem for change (using accurate > >>>>>>> backups) the director times out. > >>>>>>> > >>>>>>> Any thoughts, or opinions? Workarounds etc? I have checked to make > >>>>>>> sure > >>>>>>> that SSSD caching is enabled, and functional. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> -Erinn > >>>>>> > >>>>>> I am assuming that these are all running as local users. > >>>>>> > >>>>>> From the sssd.conf man page in the nss section: > >>>>>> > >>>>>> filter_users, filter_groups (string) > >>>>>> Exclude certain users from being fetched from the sss NSS > >>>>>> database. This is particularly useful for system accounts. This option > >>>>>> can also be set per-domain or include fully-qualified names to filter > >>>>>> only users from the > >>>>>> particular domain. > >>>>>> > >>>>>> Default: root > >>>>>> > >>>>>> > >>>>>> Try adding this to your sssd.conf: > >>>>>> > >>>>>> [nss] > >>>>>> filter_groups = root,bacula,aide,otherdaemonuser <-as needed > >>>>>> filter_users = root,bacula,aide,otherdaemonuser <- as needed > >>>>>> > >>>>>> Let me know if that solves your issue. > >>>>>> > >>>>> > >>>>> Thanks for pointing that out, completely missed that option! Wouldn't it > >>>>> be sweet to have an option that say looked at /etc/login.defs and just > >>>>> didn't lookup anything under MIN_UID, on the assumption that those are > >>>>> system accounts? Certainly would stop a lot of lookups I imagine. > >>>> > >>>> We already have range options (min_id/max_id), but unfortunately that > >>>> doesn't help when an application asks for information by name. > >>>> You either permanently blacklist such name or you have to do the lookup > >>>> and then find out it either a) does not exist, or b) it has to be > >>>> filtered out. > >>>> > >>>>> Of course you would have to leave it as an option and probably default > >>>>> it to off given the odd things people do with their systems. > >>>> > >>>> Indeed sssd used to enforce a min id range of 1000 and we turned it off > >>>> in the default configuration due to issues with weird configurations. > >>>> > >>>> Can you try using both min_id and filter_users and see if it makes any > >>>> difference in your case ? > >>>> > >>>> Simo. > >>>> > >>> > >>> Even when performing getpwuid() calls, SSSD first looks up the user > >>> information, reads the UID LDAP attribute and then checks the UID value > >>> from LDAP against min_id/max_id values. > >> > >> Not according to my reading of the sources, if you look into > >> nss_cmd_getpwuid_search() you'll see that we proceed only if we first > >> pass the min_id/max_id range check, otherwise we return ENOENT. > >> > >> Simo. > > > > Sorry, you're right and I need to warm up my brain a little more after > > the Christmas break. > > > > Thanks! > > I am going through some testing now to try and get you folks something > more definitive. However, from an early test adding users/groups to > filter_* seemed to reduce the performance hit slightly, but it did not > take it anywhere near the levels it was at before sssd was in place. > > Like I said I will continue to test and get you folks some more > definitive results, probably later today. Thanks for all the info and > feedback.
Hi Erinn, can you please tell what's the baseline you are comparing against ? Is it nss_ldap ? With or without nscd ? Simo. -- Simo Sorce * Red Hat, Inc * New York _______________________________________________ Freeipa-users mailing list [email protected] https://www.redhat.com/mailman/listinfo/freeipa-users
