Hi Sumit,
See, I told you I'd get it wrong!
Yes, things like id are properly evil, and an easy way to see the difference in
performance between ignore_group_members true/false.
I'll go back to hiding in my corner ;)
Thanks,
John
--
John Hodrien (he/him)
Principal Teaching and Research Support Specialist, School of Computer Science
2.22 Bragg Building, University of Leeds
________________________________
From: Sumit Bose <sb...@redhat.com>
Sent: 23 July 2025 10:44
To: End-user discussions about the System Security Services Daemon
<sssd-users@lists.fedorahosted.org>
Cc: Christopher Paul <chris.p...@rexconsulting.net>; John Hodrien
<j.h.hodr...@leeds.ac.uk>
Subject: Re: [SSSD-users]Re: SSSD with rfc2307bis causes thousands of
concurrent LDAP queries, triggering OpenLDAP flow control
CAUTION: External Message. Use caution opening links and attachments.
Am Wed, Jul 23, 2025 at 08:35:17AM +0000 schrieb John Hodrien via sssd-users:
> Again, not the expert here and there's likely some wrongness in what I'm
> saying, but...
>
> I think the problem is the legacy (and still very much used) functions for
> gathering group information work on the basis that a big sweep of the groups
> file gathering all the information is much more sensible than lots of
> targeted pokes for just what you need. So you have things like this,
> directly quoting from the man pages:
>
> The initgroups() function initializes the group access list by reading the
> group database /etc/group and using all groups of which user is a member.
>
> The getgrouplist() function scans the group database (see
> group(5)) to obtain the list of groups that user belongs to. Up
> to *ngroups of these groups are returned in the array groups.
>
> struct group {
> char *gr_name; /* group name */
> char *gr_passwd; /* group password */
> gid_t gr_gid; /* group ID */
> char **gr_mem; /* NULL-terminated array of pointers
> to names of group members */
> };
>
Hi,
where did you get this information? The `getgrouplist()` function is
defined as
int getgrouplist(const char *user, gid_t group, gid_t *groups, int
*ngroups);
and returns only a list of GIDs and not the full group structs.
Where possible we try to avoid to lookup group-members internally. But a
typical use case for the `getgrouplist()` call is the `id` command and
if you do not use `id` with the `-g` option `id` will call `getgrgid()`
for all GIDs returned and this has to return the group struct with all
members.
Btw, I think the original question was not about the large number of
requests in general but that SSSD is sending those in parallel (sending
the next request before a reply for the previous was received). This is
indeed unexpected and comes most probably from
`sdap_process_group_members_2307bis()`
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSSSD%2Fsssd%2Fblob%2Fmaster%2Fsrc%2Fproviders%2Fldap%2Fsdap_async_groups.c%23L1382&data=05%7C02%7Cj.h.hodrien%40leeds.ac.uk%7Ceab2e400cca147d4420708ddc9cd8c2c%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C0%7C0%7C638888606840281033%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=lOPdYXpkGjjLOpSxxqtXqeYO7x%2FowgnoicZVCBM1f34%3D&reserved=0<https://github.com/SSSD/sssd/blob/master/src/providers/ldap/sdap_async_groups.c#L1382>
As you can see there is a loop which calls
`sdap_process_missing_member_2307bis()` for every group missing in the
cache which sends an asynchronous LDAP search without waiting for a
reply. This should most probably be handled differently to not overload
and LDAP server. Can you open an issue on github for this?
bye,
Sumit
> So you use initgroups to find out about all the groups a user is a member of,
> and in doing that you find out about all those groups, which means you need
> to populate the group members of all those groups. Bleurgh! This is what
> leads to the performance fun.
>
> Say I'm restricting access to a service on the basis of group membership. I
> can either ask "Which groups is this user a member of?", then check that the
> group I want is there, or I can ask "Who is a member of this group?". The
> former means full group enumeration of every group a user is a member of,
> whereas the latter involves just a single group, but it's not all positive.
>
> Using the ignore_group_members option in SSSD works around this and makes
> things much faster, but there is a downside. If I go the latter route, and
> we've used ignore_group_members, then when I query a group, it tells me that
> nobody is a member of it.
>
> So "What groups is Sally a member of? People". "Who is a member of People?
> Nobody". That then can cause you some grief with ignore_group_members. In
> the past I know newgrp would fail to work properly with this configuration.
>
> But to be honest, these niggles are in my experience few and far between.
> With a large LDAP setup, ignore_group_members is an essential performance
> boost.
>
> John
>
> --
> John Hodrien (he/him)
> Principal Teaching and Research Support Specialist, School of Computer Science
> 2.22 Bragg Building, University of Leeds
> ________________________________
> From: Christopher Paul via sssd-users <sssd-users@lists.fedorahosted.org>
> Sent: 23 July 2025 00:52
> To: sssd-users@lists.fedorahosted.org <sssd-users@lists.fedorahosted.org>
> Cc: Christopher Paul <chris.p...@rexconsulting.net>
> Subject: [SSSD-users]Re: SSSD with rfc2307bis causes thousands of concurrent
> LDAP queries, triggering OpenLDAP flow control
>
>
> CAUTION: External Message. Use caution opening links and attachments.
>
>
> On 7/22/25 16:48, Christopher Paul via sssd-users wrote:
>
>
> On 7/22/25 14:53, John Hodrien wrote:
> [...]
>
> That's my understanding anyway, but I'm not an expert.
>
>
> You're probably right, but it would be nice if SSSD could retrieve groups for
> a user but not look up all the members attribute values for all the other
> members of each group to which one user belongs.
>
>
> I notice that with "ignore_group_members = true", that the "groups <uid>"
> command still works.
>
> Oh blast it. Actually with "ignore_group_members = true", "id <uid>" also
> works, and it does not look up all the members attribute values for all the
> other members of each group to which one user belongs. So this feature does
> exist and it's called "ignore_group_members = true".
>
>
> But I guess it would be nice if "getent group <group name>" worked to list
> all members without looking up their user values. I see no reason why a group
> lookup should cause all members' attribute values to be looked up.
>
>
> Chris
> --
> _______________________________________________
> sssd-users mailing list -- sssd-users@lists.fedorahosted.org
> To unsubscribe send an email to sssd-users-le...@lists.fedorahosted.org
> Fedora Code of Conduct:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.fedoraproject.org%2Fen-US%2Fproject%2Fcode-of-conduct%2F&data=05%7C02%7Cj.h.hodrien%40leeds.ac.uk%7Ceab2e400cca147d4420708ddc9cd8c2c%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C0%7C0%7C638888606840311048%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=03fZnX6%2BM7%2BqdyzDkMedS2VakvpkiPiJIB2%2FPA2QrvY%3D&reserved=0<https://docs.fedoraproject.org/en-US/project/code-of-conduct/>
> List Guidelines:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffedoraproject.org%2Fwiki%2FMailing_list_guidelines&data=05%7C02%7Cj.h.hodrien%40leeds.ac.uk%7Ceab2e400cca147d4420708ddc9cd8c2c%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C0%7C0%7C638888606840332714%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0hm7pWTDpNAmmTYgH%2FCOOHliquBNQHixpoFt52XsBck%3D&reserved=0<https://fedoraproject.org/wiki/Mailing_list_guidelines>
> List Archives:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.fedorahosted.org%2Farchives%2Flist%2Fsssd-users%40lists.fedorahosted.org&data=05%7C02%7Cj.h.hodrien%40leeds.ac.uk%7Ceab2e400cca147d4420708ddc9cd8c2c%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C0%7C0%7C638888606840348744%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0g2avAQQ3hxeJSjS3isJAEgNBY%2F%2FVKqxF4HXmJewyEY%3D&reserved=0<https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org>
> Do not reply to spam, report it:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpagure.io%2Ffedora-infrastructure%2Fnew_issue&data=05%7C02%7Cj.h.hodrien%40leeds.ac.uk%7Ceab2e400cca147d4420708ddc9cd8c2c%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C0%7C0%7C638888606840367546%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=5OUFQiukMmckb951jHr2Jlawnt65UgAJopzVc33iMcQ%3D&reserved=0<https://pagure.io/fedora-infrastructure/new_issue>
--
_______________________________________________
sssd-users mailing list -- sssd-users@lists.fedorahosted.org
To unsubscribe send an email to sssd-users-le...@lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
