So the idea to turn on debug_level = 9 on the client and view the logs was inspired. We turned on debug level 9 on 4 clients;
2 in the list (that we got from AD team of servers in that AMERAustin site hitting the non-AMER Austin AD DCs). 2 not in their list. (1 in another AMER site). Consistently, we see them querying the rootDSE for all these domains on the hour and the half-hour. Querying the local AMER rootDSE in Austin is not a problem; they have beaucoup AD DCs in Austin. Querying the other domains' rootDSEs in Austin is a problem; they typically only have 2 AD DCs from each region. Here’s an example from the client logs. First client: (2023-10-03 0:30:06): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x1000): New LDAP connection to [ldap://ausdc16corp05.corp.com:389/??base] with fd [30]. (2023-10-03 0:30:06): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): Getting rootdse … (2023-10-03 0:41:18): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x1000): New LDAP connection to [ldap:// AUSDC16ROAMER01.amer.corp.com:389/??base] with fd [20]. (2023-10-03 0:41:18): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): Getting rootdse Another client: (2023-10-02 11:30:02): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x4000): [RID#48] New connection to [ldap:// ausdc16amer33.amer.corp.com:3268/??base] with fd [25] (2023-10-02 11:30:02): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): [RID#48] Getting rootdse -- (2023-10-02 11:30:04): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x4000): [RID#49] New connection to [ldap:// ausdc16emea05.emea.corp.com:389/??base] with fd [26] (2023-10-02 11:30:04): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): [RID#49] Getting rootdse -- (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x4000): [RID#50] New connection to [ldap:// ausdc16apac06.apac.corp.com:389/??base] with fd [27] (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): [RID#50] Getting rootdse -- (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x4000): [RID#51] New connection to [ldap:// AUSDC16JAPN02.japn.corp.com:389/??base] with fd [28] (2023-10-02 11:30:05): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): [RID#51] Getting rootdse -- (2023-10-02 11:32:52): [be[amer.corp.com]] [sdap_ldap_connect_callback_add] (0x4000): [RID#84] New connection to [ldap:// ausdc16emea05.emea.corp.com:389/??base] with fd [26] (2023-10-02 11:32:52): [be[amer.corp.com]] [sdap_get_rootdse_send] (0x4000): [RID#84] Getting rootdse -- BTW, this seems to occurs on both RHEL7 and RHEL8. (Haven't looked at our RHEL9 builds yet). It's occurring on all servers to all rootDSEs, but only a problem for AMERAustin, since Austin is such a heavily-populated. These rootDSEs change almost never. Any way to have it query not as frequently, or randomize when servers query these rootDSEs. Spike On Mon, Oct 2, 2023 at 2:37 AM Alexey Tikhonov <[email protected]> wrote: > Hi, > > On Mon, Oct 2, 2023 at 6:20 AM Spike White <[email protected]> wrote: > >> All, >> >> Is there anything in sssd's RHEL and RHEL-like Linux server OS settings >> that perform LDAP binds or connections to AD every 30 minutes? >> >> What our AD team is seeing is all of the DCs in our biggest AMER AD site >> peak with LDAP sessions for about 10 minutes at the top of the hour then >> again at the bottom of the hour. No other AD site in the world appears to >> see this behavior not even other AD sites in this metro area. >> >> The reason they noticed is that our non-amer DCs in this biggest AD site >> hit their 5k LDAP client session limit during those 10 minutes every 30 >> minutes. Meaning any clients attempting to establish a LDAP session past >> 5000 are dropped by the DC. In their research they see thousands LDAP >> Binds by RHEL Linux servers against two specific non-AMER AD DCs in a short >> period of time after digging through some LDAP log samples that they pulled >> from these DCs. >> > > Can they also say what operations are being performed by those connections? > Or can you check SSSD logs on the client side? > > I wonder if this could be `ldap_sudo_smart_refresh_interval`... > > > >> >> In this major AD sites, we have dozens and dozens of AMER AD DCs. So >> there's enough preferred AD DCs to spread the load. But typically for the >> non-AMER regions, the AD team puts 2 of each regions DCs in a site. For >> instance, for APAC they would be put two APAC DCs in this AMER major site. >> Thus all AMER RHEL servers in this site would randomly hit dozens of AMER >> DCs, but concentrate on these two preferred APAC DCs. (preferred because >> they're in this locatiion). >> >> I know our older AD integration product used to hit AD every 30 mins to >> check GPOs, but we're not implementing GPOs with sssd. >> >> Spike >> _______________________________________________ >> sssd-users mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Fedora Code of Conduct: >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: >> https://lists.fedorahosted.org/archives/list/[email protected] >> Do not reply to spam, report it: >> https://pagure.io/fedora-infrastructure/new_issue >> > _______________________________________________ > sssd-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/[email protected] > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue >
_______________________________________________ sssd-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
