On Mon, Oct 2, 2023 at 7:01 PM Spike White <[email protected]> wrote: > > So the idea to turn on debug_level = 9 on the client and view the logs was > inspired. We turned on debug level 9 on 4 clients; > > 2 in the list (that we got from AD team of servers in that AMERAustin site > hitting the non-AMER Austin AD DCs). > 2 not in their list. (1 in another AMER site). > > Consistently, we see them querying the rootDSE for all these domains on the > hour and the half-hour. Querying the local AMER rootDSE in Austin is not a > problem; they have beaucoup AD DCs in Austin. Querying the other domains' > rootDSEs in Austin is a problem; they typically only have 2 AD DCs from each > region. > > Here’s an example from the client logs. First client: > > > > (2023-10-03 0:30:06): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x1000): New LDAP connection to [ldap://ausdc16corp05.corp.com:389/??base] > with fd [30]. > > (2023-10-03 0:30:06): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > Getting rootdse > > … > > (2023-10-03 0:41:18): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x1000): New LDAP connection to > [ldap://AUSDC16ROAMER01.amer.corp.com:389/??base] with fd [20]. > > (2023-10-03 0:41:18): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > Getting rootdse > > > > Another client: > > > > (2023-10-02 11:30:02): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x4000): [RID#48] New connection to > [ldap://ausdc16amer33.amer.corp.com:3268/??base] with fd [25] > > (2023-10-02 11:30:02): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > [RID#48] Getting rootdse >
What goes next for this 'RID#48', after 'rootdse' is read? SSSD doesn't query 'rootdse' on its own. It is being read (as a first operation) when a new connection is established. You need to see what happens next to figure out *why* this new connection is established. As a practical side note, you can also increase `ldap_connection_expire_timeout` to keep connections longer. But I would figure out the reason first. > -- > > (2023-10-02 11:30:04): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x4000): [RID#49] New connection to > [ldap://ausdc16emea05.emea.corp.com:389/??base] with fd [26] > > (2023-10-02 11:30:04): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > [RID#49] Getting rootdse > > -- > > (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x4000): [RID#50] New connection to > [ldap://ausdc16apac06.apac.corp.com:389/??base] with fd [27] > > (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > [RID#50] Getting rootdse > > -- > > (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x4000): [RID#51] New connection to > [ldap://AUSDC16JAPN02.japn.corp.com:389/??base] with fd [28] > > (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > [RID#51] Getting rootdse > > -- > > (2023-10-02 11:32:52): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] > (0x4000): [RID#84] New connection to > [ldap://ausdc16emea05.emea.corp.com:389/??base] with fd [26] > > (2023-10-02 11:32:52): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): > [RID#84] Getting rootdse > > -- > > > > BTW, this seems to occurs on both RHEL7 and RHEL8. (Haven't looked at our > RHEL9 builds yet). It's occurring on all servers to all rootDSEs, but only > a problem for AMERAustin, since Austin is such a heavily-populated. > > > These rootDSEs change almost never. Any way to have it query not as > frequently, or randomize when servers query these rootDSEs. > > > Spike > > On Mon, Oct 2, 2023 at 2:37 AM Alexey Tikhonov <[email protected]> wrote: >> >> Hi, >> >> On Mon, Oct 2, 2023 at 6:20 AM Spike White <[email protected]> wrote: >>> >>> All, >>> >>> Is there anything in sssd's RHEL and RHEL-like Linux server OS settings >>> that perform LDAP binds or connections to AD every 30 minutes? >>> >>> What our AD team is seeing is all of the DCs in our biggest AMER AD site >>> peak with LDAP sessions for about 10 minutes at the top of the hour then >>> again at the bottom of the hour. No other AD site in the world appears to >>> see this behavior not even other AD sites in this metro area. >>> >>> The reason they noticed is that our non-amer DCs in this biggest AD site >>> hit their 5k LDAP client session limit during those 10 minutes every 30 >>> minutes. Meaning any clients attempting to establish a LDAP session past >>> 5000 are dropped by the DC. In their research they see thousands LDAP >>> Binds by RHEL Linux servers against two specific non-AMER AD DCs in a short >>> period of time after digging through some LDAP log samples that they pulled >>> from these DCs. >> >> >> Can they also say what operations are being performed by those connections? >> Or can you check SSSD logs on the client side? >> >> I wonder if this could be `ldap_sudo_smart_refresh_interval`... >> >> >>> >>> >>> In this major AD sites, we have dozens and dozens of AMER AD DCs. So >>> there's enough preferred AD DCs to spread the load. But typically for the >>> non-AMER regions, the AD team puts 2 of each regions DCs in a site. For >>> instance, for APAC they would be put two APAC DCs in this AMER major site. >>> Thus all AMER RHEL servers in this site would randomly hit dozens of AMER >>> DCs, but concentrate on these two preferred APAC DCs. (preferred because >>> they're in this locatiion). >>> >>> I know our older AD integration product used to hit AD every 30 mins to >>> check GPOs, but we're not implementing GPOs with sssd. >>> >>> Spike >>> _______________________________________________ >>> sssd-users mailing list -- [email protected] >>> To unsubscribe send an email to [email protected] >>> Fedora Code of Conduct: >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: >>> https://lists.fedorahosted.org/archives/list/[email protected] >>> Do not reply to spam, report it: >>> https://pagure.io/fedora-infrastructure/new_issue >> >> _______________________________________________ >> sssd-users mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Fedora Code of Conduct: >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: >> https://lists.fedorahosted.org/archives/list/[email protected] >> Do not reply to spam, report it: >> https://pagure.io/fedora-infrastructure/new_issue > > _______________________________________________ > sssd-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/[email protected] > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue _______________________________________________ sssd-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
