I have an inherited IPA domain that is a subdomain of an active directory domain, e.g. ipa.ad1.com as a child of ad1.com. The IPA domain has AD Trust enabled and a one way domain trust to another AD sub domain, e.g. we want to use user logins from the AD domain users.ad2.com which is a child domain of ad2.com. We are also using AD security group from the user.ad2.com domain to apply group based access control. e.g. we are using simple authentication on SSSD to limit who can login and using AD groups to define sudo access. This users domain and AD servers is managed by another team.
Everything was working for some time and then we started seeing intermittent problems with authentication, a quick restart of the IPA server would resolve the problem temporarily, but then it would stop again. Even if we could login using SSH keys the sudo access would not work, it would appear to lose group membership details. I have recently updated all of the IPA nodes to RHEL9 and made sure that DNS is updated correctly. The sssd.conf configuration on the IPA server looks as follows [domain/ipa.ad1.com] debug_level = 6 id_provider = ipa ipa_server = ipa-3.ipa.ad1.com ipa_domain = ipa.ad1.com ipa_hostname = ipa-3.ipa.ad1.com auth_provider = ipa chpass_provider = ipa access_provider = ipa cache_credentials = True ldap_tls_cacert = /etc/ipa/ca.crt krb5_store_password_if_offline = True sudo_provider = ipa autofs_provider = ipa subdomains_provider = ipa session_provider = ipa hostid_provider = ipa ipa_server_mode = True subdomain_homedir = /home/%u default_shell = /bin/bash override_shell = /bin/bash [sssd] services = nss, pam, sudo, ifp domains = ipa.ad1.com domain_resolution_order = users.ad2.com [nss] homedir_substring = /home [pam] [sudo] [autofs] [ssh] [pac] [ifp] allowed_uids = ipaapi, root [session_recording] I have debug level 6 enabled on SSSD and when I check the domain status I see the following more often than not. The ad2.com forest domains are offline. They go online and then as soon as someone tries to login again then either both or just the users.ad2.com domain go offline which causes the login to fail. ipa.ad1.com Online status: Online ad1.com Online status: Online ad2.com Online status: Offline users.ad2.com Online status: Offline When I look at the SSSD domain logs I see the following (I have replaced internal domain names or hostname) * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sbus_senders_lookup] (0x2000): Looking for identity of sender [sssd.ifp] * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sss_domain_get_state] (0x1000): Domain ipa.ad1.com is Active * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sss_domain_get_state] (0x1000): Domain ad1.com is Active * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sss_domain_get_state] (0x1000): Domain AD2.COM is Active * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sbus_issue_request_done] (0x0400): sssd.DataProvider.Backend.IsOnline: Success * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sbus_dispatch] (0x4000): Dispatching. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [_read_pipe_handler] (0x0400): [RID#1162] EOF received, client finished * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_tgt_recv] (0x0400): [RID#1162] Child responded: 0 [FILE:/var/lib/sss/db/ccache_AD2.COM], expired on [1686187555] * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_auth_step] (0x0100): [RID#1162] expire timeout is 900 * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_auth_step] (0x1000): [RID#1162] the connection will expire at 1686152455 * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0100): [RID#1162] Executing sasl bind mech: GSSAPI, user: AUTH$ * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0020): [RID#1162] ldap_sasl_interactive_bind_s failed (-2)[Local error] ********************** BACKTRACE DUMP ENDS HERE ********************************* (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0080): [RID#1162] Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [child_sig_handler] (0x0100): [RID#1162] child [9519] finished successfully. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_connect_recv] (0x0040): [RID#1162] Unable to establish connection [1432158227]: Authentication Failed ********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE: * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0080): [RID#1162] Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)] * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [child_sig_handler] (0x1000): [RID#1162] Waiting for child [9519]. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [child_sig_handler] (0x0100): [RID#1162] child [9519] finished successfully. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_connect_recv] (0x0040): [RID#1162] Unable to establish connection [1432158227]: Authentication Failed ********************** BACKTRACE DUMP ENDS HERE ********************************* (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 3268 of server 'ad1-dc-1.ad1.com' as 'not working' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 3268 of duplicate server 'ad1-dc-1.ad1.com' as 'not working' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_gc_AD2.COM' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolve_srv_send] (0x0200): [RID#1162] The status of SRV lookup is resolved (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_files_send] (0x0100): [RID#1162] Trying to resolve A record of 'ad1-dc-3.ad1.com' in files (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'ad1-dc-3.ad1.com' as 'resolving name' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_files_send] (0x0100): [RID#1162] Trying to resolve AAAA record of 'ad1-dc-3.ad1.com' in files (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_next] (0x0200): [RID#1162] No more address families to retry (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_dns_query] (0x0100): [RID#1162] Trying to resolve A record of 'ad1-dc-3.ad1.com' in DNS (2023-06-07 16:25:55): [be[ipa.ad1.com]] [request_watch_destructor] (0x0400): [RID#1162] Deleting request watch (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'ad1-dc-3.ad1.com' as 'name resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_resolve_server_process] (0x0200): [RID#1162] Found address for server ad1-dc-3.ad1.com: [172.28.8.7] TTL 3600 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ad_resolve_callback] (0x0100): [RID#1162] Constructed uri 'ldap://ad1-dc-3.ad1.com' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ad_resolve_callback] (0x0100): [RID#1162] Constructed GC uri 'ldap://ad1-dc-3.ad1.com:3268' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sssd_async_socket_init_send] (0x0400): [RID#1162] Setting 6 seconds timeout [ldap_network_timeout] for connecting (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_generic_ext_step] (0x0400): [RID#1162] calling ldap_search_ext with [(objectclass=*)][]. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_generic_op_finished] (0x0400): [RID#1162] Search result: Success(0), no errmsg set (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_server_opts_from_rootdse] (0x0100): [RID#1162] Setting AD compatibility level to [7] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_server_opts_from_rootdse] (0x0100): [RID#1162] Will look for schema at [CN=Schema,CN=Configuration,DC=ad1,DC=com] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_kinit_send] (0x0400): [RID#1162] Attempting kinit (/var/lib/sss/keytabs/AD2.COM.keytab, AUTH$, AD2.COM, 86400) (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_AD2.COM' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolve_srv_send] (0x0200): [RID#1162] The status of SRV lookup is resolved (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_resolve_server_process] (0x0200): [RID#1162] Found address for server dcc01.ad2.com: [10.194.34.10] TTL 2107 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [create_tgt_req_send_buffer] (0x0400): [RID#1162] buffer size: 73 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_tgt_child_timeout] (0x0400): [RID#1162] Setting 8 seconds timeout for TGT child (2023-06-07 16:25:55): [be[ipa.ad1.com]] [_write_pipe_handler] (0x0400): [RID#1162] All data has been sent! (2023-06-07 16:25:55): [be[ipa.ad1.com]] [_read_pipe_handler] (0x0400): [RID#1162] EOF received, client finished (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_tgt_recv] (0x0400): [RID#1162] Child responded: 0 [FILE:/var/lib/sss/db/ccache_AD2.COM], expired on [1686187555] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_auth_step] (0x0100): [RID#1162] expire timeout is 900 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0100): [RID#1162] Executing sasl bind mech: GSSAPI, user: AUTH$ (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0020): [RID#1162] ldap_sasl_interactive_bind_s failed (-2)[Local error] * ... skipping repetitive backtrace ... (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0080): [RID#1162] Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [child_sig_handler] (0x0100): [RID#1162] child [9522] finished successfully. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_connect_recv] (0x0040): [RID#1162] Unable to establish connection [1432158227]: Authentication Failed * ... skipping repetitive backtrace ... (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 3268 of server 'ad1-dc-3.ad1.com' as 'not working' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 3268 of duplicate server 'ad1-dc-3.ad1.com' as 'not working' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_gc_AD2.COM' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolve_srv_send] (0x0200): [RID#1162] The status of SRV lookup is resolved (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_files_send] (0x0100): [RID#1162] Trying to resolve A record of 'ad1-dc-2.ad1.com' in files (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'ad1-dc-2.ad1.com' as 'resolving name' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_files_send] (0x0100): [RID#1162] Trying to resolve AAAA record of 'ad1-dc-2.ad1.com' in files (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_next] (0x0200): [RID#1162] No more address families to retry (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolv_gethostbyname_dns_query] (0x0100): [RID#1162] Trying to resolve A record of 'ad1-dc-2.ad1.com' in DNS (2023-06-07 16:25:55): [be[ipa.ad1.com]] [request_watch_destructor] (0x0400): [RID#1162] Deleting request watch (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'ad1-dc-2.ad1.com' as 'name resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_resolve_server_process] (0x0200): [RID#1162] Found address for server ad1-dc-2.ad1.com: [172.28.8.6] TTL 3600 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ad_resolve_callback] (0x0100): [RID#1162] Constructed uri 'ldap://ad1-dc-2.ad1.com' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ad_resolve_callback] (0x0100): [RID#1162] Constructed GC uri 'ldap://ad1-dc-2.ad1.com:3268' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sssd_async_socket_init_send] (0x0400): [RID#1162] Setting 6 seconds timeout [ldap_network_timeout] for connecting (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_generic_ext_step] (0x0400): [RID#1162] calling ldap_search_ext with [(objectclass=*)][]. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_generic_op_finished] (0x0400): [RID#1162] Search result: Success(0), no errmsg set (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_server_opts_from_rootdse] (0x0100): [RID#1162] Setting AD compatibility level to [7] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_server_opts_from_rootdse] (0x0100): [RID#1162] Will look for schema at [CN=Schema,CN=Configuration,DC=ad1,DC=com] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_kinit_send] (0x0400): [RID#1162] Attempting kinit (/var/lib/sss/keytabs/AD2.COM.keytab, AUTH$, AD2.COM, 86400) (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_AD2.COM' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [resolve_srv_send] (0x0200): [RID#1162] The status of SRV lookup is resolved (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_resolve_server_process] (0x0200): [RID#1162] Found address for server dcc01.ad2.com: [10.194.34.10] TTL 2107 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [create_tgt_req_send_buffer] (0x0400): [RID#1162] buffer size: 73 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_tgt_child_timeout] (0x0400): [RID#1162] Setting 8 seconds timeout for TGT child (2023-06-07 16:25:55): [be[ipa.ad1.com]] [_write_pipe_handler] (0x0400): [RID#1162] All data has been sent! (2023-06-07 16:25:55): [be[ipa.ad1.com]] [_read_pipe_handler] (0x0400): [RID#1162] EOF received, client finished (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_get_tgt_recv] (0x0400): [RID#1162] Child responded: 0 [FILE:/var/lib/sss/db/ccache_AD2.COM], expired on [1686187555] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_auth_step] (0x0100): [RID#1162] expire timeout is 900 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0100): [RID#1162] Executing sasl bind mech: GSSAPI, user: AUTH$ (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0020): [RID#1162] ldap_sasl_interactive_bind_s failed (-2)[Local error] * ... skipping repetitive backtrace ... (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sasl_bind_send] (0x0080): [RID#1162] Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server not found in Kerberos database)] (2023-06-07 16:25:55): [be[ipa.ad1.com]] [child_sig_handler] (0x0100): [RID#1162] child [9523] finished successfully. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_cli_connect_recv] (0x0040): [RID#1162] Unable to establish connection [1432158227]: Authentication Failed * ... skipping repetitive backtrace ... (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 3268 of server 'ad1-dc-2.ad1.com' as 'not working' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 3268 of duplicate server 'ad1-dc-2.ad1.com' as 'not working' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_gc_AD2.COM' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0020): [RID#1162] No available servers for service 'sd_gc_AD2.COM' ********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE: * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [_be_fo_set_port_status] (0x8000): [RID#1162] Setting status: PORT_NOT_WORKING. Called from: src/providers/ldap/sdap_async_connection.c: sdap_cli_connect_recv: 2136 * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 3268 of server 'ad1-dc-2.ad1.com' as 'not working' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 3268 of duplicate server 'ad1-dc-2.ad1.com' as 'not working' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_handle_release] (0x2000): [RID#1162] Trace: sh[0x556c865496d0], connected[1], ops[(nil)], ldap[0x556c865f89b0], destructor_lock[0], release_memory[0] * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [remove_connection_callback] (0x4000): [RID#1162] Successfully removed connection callback. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_id_op_connect_done] (0x4000): [RID#1162] attempting failover retry on op #1 * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_id_op_connect_step] (0x4000): [RID#1162] beginning to connect * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_gc_AD2.COM' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_server_status] (0x1000): [RID#1162] Status of server 'ad1-dc-1.ad1.com' is 'name resolved' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x1000): [RID#1162] Port status of port 3268 for server 'ad1-dc-1.ad1.com' is 'not working' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_server_status] (0x1000): [RID#1162] Status of server 'ad1-dc-3.ad1.com' is 'name resolved' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x1000): [RID#1162] Port status of port 3268 for server 'ad1-dc-3.ad1.com' is 'not working' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_server_status] (0x1000): [RID#1162] Status of server 'ad1-dc-2.ad1.com' is 'name resolved' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x1000): [RID#1162] Port status of port 3268 for server 'ad1-dc-2.ad1.com' is 'not working' * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0020): [RID#1162] No available servers for service 'sd_gc_AD2.COM' ********************** BACKTRACE DUMP ENDS HERE ********************************* (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_id_op_connect_done] (0x0400): [RID#1162] Failed to connect to server, but ignore mark offline is enabled. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_srv_ad_acct_lookup_done] (0x0080): [RID#1162] Subdomain lookup failed, will try to reset subdomain. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_server_trusted_dom_setup_1way] (0x0400): [RID#1162] Will re-fetch keytab for USERS.AD2.COM (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_getkeytab_send] (0x0400): [RID#1162] Retrieving keytab for [email protected] from ipa-3.ipa.ad1.com into /var/lib/sss/keytabs/AD2.COM.keytabIw4R58 using ccache /var/lib/sss/db/ccache_AUTH.SSDIS.LOC (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sbus_issue_request_done] (0x0400): sssd.DataProvider.Backend.IsOnline: Success (2023-06-07 16:25:55): [be[ipa.ad1.com]] [child_sig_handler] (0x0100): [RID#1162] child [9524] finished successfully. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_server_trust_1way_kt_done] (0x0400): [RID#1162] Keytab successfully retrieved to /var/lib/sss/keytabs/AD2.COM.keytabIw4R58 (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_server_trust_1way_kt_done] (0x0400): [RID#1162] Keytab /var/lib/sss/keytabs/AD2.COM.keytabIw4R58 contains the expected principals (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_server_trust_1way_kt_done] (0x0400): [RID#1162] Established trust context for USERS.AD2.COM (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_srv_ad_acct_retried] (0x0400): [RID#1162] Subdomain re-set, will retry lookup (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'dcf01.users.ad2.com' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'dcf01.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'dcf01.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'DCC02.USERS.AD2.COM' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'DCC02.USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'DCC02.USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'DCC02.USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'dcs02.users.ad2.com' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'dcs02.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'dcs02.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'dcf02.users.ad2.com' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'dcf02.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'dcf02.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'dcc01.users.ad2.com' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'dcc01.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'dcc01.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'DCC02.USERS.AD2.COM' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'DCC02.USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'DCC02.USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'DCC02.USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_server_common_status] (0x0100): [RID#1162] Marking server 'dcs03.users.ad2.com' as 'name not resolved' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 389 of server 'dcs03.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0400): [RID#1162] Marking port 389 of duplicate server 'dcs03.users.ad2.com' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [set_srv_data_status] (0x0100): [RID#1162] Marking SRV lookup of service 'sd_gc_USERS.AD2.COM' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_set_port_status] (0x0100): [RID#1162] Marking port 0 of server '(no name)' as 'neutral' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_srv_ad_acct_lookup_step] (0x0400): [RID#1162] Looking up AD account (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0100): [RID#1162] Trying to resolve service 'sd_gc_AD2.COM' (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [get_port_status] (0x0080): [RID#1162] SSSD is unable to complete the full connection request, this internal status does not necessarily indicate network port issues. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [fo_resolve_service_send] (0x0020): [RID#1162] No available servers for service 'sd_gc_AD2.COM' * ... skipping repetitive backtrace ... (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_id_op_connect_done] (0x0400): [RID#1162] Failed to connect to server, but ignore mark offline is enabled. (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_srv_ad_acct_lookup_done] (0x0040): [RID#1162] ipa_get_*_acct request failed: [1432158276]: Subdomain is inactive. ********************** PREVIOUS MESSAGE WAS TRIGGERED BY THE FOLLOWING BACKTRACE: * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_resolve_server_done] (0x1000): [RID#1162] Server [NULL] resolution failed: [5]: Input/output error * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_id_op_connect_done] (0x0400): [RID#1162] Failed to connect to server, but ignore mark offline is enabled. * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [sdap_id_op_connect_done] (0x4000): [RID#1162] notify error to op #1: 5 [Input/output error] * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_mark_dom_offline] (0x1000): [RID#1162] Marking subdomain USERS.AD2.COM offline * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [be_mark_subdom_offline] (0x1000): [RID#1162] Marking subdomain USERS.AD2.COM as inactive * (2023-06-07 16:25:55): [be[ipa.ad1.com]] [ipa_srv_ad_acct_lookup_done] (0x0040): [RID#1162] ipa_get_*_acct request failed: [1432158276]: Subdomain is inactive. ********************** BACKTRACE DUMP ENDS HERE ********************************* As far as I can tell the users.ad2.com domain is "Active", but the bind fails to the domain which forces a domain check to take place. Ultimately, this fails and the domain is flagged offline so the authentication fails. There seems to be some odd lines where the users.ad2.com validation is trying to connect to servers from the ad1.com domain and global catalog validation which is failing. Not sure why this would be taking place. This seems to culminate in the line (2023-06-07 16:25:55): [be[auth.ssdis.loc]] [fo_resolve_service_send] (0x0020): [RID#1162] No available servers for service 'sd_gc_AD2.COM' I can see actions taking place but cannot determine why they are taking place. I can kinit as a users.ad2.com user successfully. Likewise I can getent users and groups successfully and see the details correctly. Yet authentication and authorisation is failing for SSH logins Any help or guidance on resolving this problem would be appreciated. _______________________________________________ FreeIPA-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
