Hi Jakub,

I've added the output of 'sssd -i -d4' below:

On 07/28/2014 03:39 AM, Jakub Hrozek wrote:
On Sun, Jul 27, 2014 at 10:42:34PM -0400, Mark Heslin wrote:
Folks,

I just stumbled on an odd issue. I have an OpenShift deployment with 2
brokers, 2 nodes, 1 rhc client
all running RHEL 6.5. I also have 2 IPA servers (1 server, 1 replica), 1 IPA
admin (tools) client all running RHEL 7.0.
All OpenShift hosts, client and IPA client are members of IPA domain
'interop.example.com'.

After creating ssh public keys on the IPA admin client for user 'ose-admin1'
and uploading them into IPA,
I am able to ssh with the key to all IPA domain hosts as user 'ose-admin1'
except the 2 node hosts.
In looking closer at the 2 node hosts I noticed that SSSD keeps failing on
start:

# service sssd restart
Stopping sssd: cat: /var/run/sssd.pid: No such file or directory
[FAILED]
Starting sssd: [FAILED]

Starting with debug mode shows:

   [root@node1/2 ~]# sssd -d9
   (Sun Jul 27 22:12:29:527689 2014) [sssd] [check_file] (0x0400): lstat for
[/var/run/nscd/socket] failed: [2][No such file or directory].
   (Sun Jul 27 22:12:29:529293 2014) [sssd] [ldb] (0x0400):
server_sort:Unable to register control with rootdse!
   (Sun Jul 27 22:12:29:529596 2014) [sssd] [confdb_get_domain_internal]
(0x0400): No enumeration for [interop.example.com]!
   (Sun Jul 27 22:12:29:529646 2014) [sssd] [confdb_get_domain_internal]
(0x1000): pwd_expiration_warning is -1
   (Sun Jul 27 22:12:29:529686 2014) [sssd] [server_setup] (0x0040): Becoming
a daemon.
At this point sssd became a deamon and detached from the terminal, so no
more debug info was printed. Can you run sssd again, adding "-i"
(interactive) this time?

[root@node2 ~]# sssd -i -d4
(Mon Jul 28 07:25:20 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [interop.example.com]: [10] (Mon Jul 28 07:25:20 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [interop.example.com]: [60] (Mon Jul 28 07:25:20 2014) [sssd] [start_service] (0x0100): Queueing service interop.example.com for startup /usr/libexec/sssd/sssd_be: error while loading shared libraries: libcares.so.2: cannot open shared object file: No such file or directory (Mon Jul 28 07:25:20 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [interop.example.com] exited with code [127] (Mon Jul 28 07:25:20 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [interop.example.com]: [10] (Mon Jul 28 07:25:20 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [interop.example.com]: [60] (Mon Jul 28 07:25:20 2014) [sssd] [start_service] (0x0100): Queueing service interop.example.com for startup /usr/libexec/sssd/sssd_be: error while loading shared libraries: libcares.so.2: cannot open shared object file: No such file or directory (Mon Jul 28 07:25:20 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [interop.example.com] exited with code [127] (Mon Jul 28 07:25:22 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [interop.example.com]: [10] (Mon Jul 28 07:25:22 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [interop.example.com]: [60] (Mon Jul 28 07:25:22 2014) [sssd] [start_service] (0x0100): Queueing service interop.example.com for startup /usr/libexec/sssd/sssd_be: error while loading shared libraries: libcares.so.2: cannot open shared object file: No such file or directory (Mon Jul 28 07:25:22 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [interop.example.com] exited with code [127] (Mon Jul 28 07:25:25 2014) [sssd] [services_startup_timeout] (0x0020): Providers did not start in time, forcing services startup! (Mon Jul 28 07:25:25 2014) [sssd] [services_startup_timeout] (0x0100): Now starting services! (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [nss]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [nss]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service nss for startup (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [pam]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [pam]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service pam for startup (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [ssh]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [ssh]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service ssh for startup (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [pac]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [pac]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service pac for startup (Mon Jul 28 07:25:25 2014) [sssd[nss]] [monitor_common_send_id] (0x0100): Sending ID: (nss,1) (Mon Jul 28 07:25:25 2014) [sssd[pam]] [monitor_common_send_id] (0x0100): Sending ID: (pam,1) (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [pam] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [pam]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [pam]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service pam for startup (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [nss] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [nss]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [nss]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service nss for startup (Mon Jul 28 07:25:25 2014) [sssd[pac]] [monitor_common_send_id] (0x0100): Sending ID: (pac,1) (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [pac] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [pac]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [pac]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service pac for startup (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [monitor_common_send_id] (0x0100): Sending ID: (ssh,1) (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [ssh] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [ssh]: [10] (Mon Jul 28 07:25:25 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [ssh]: [60] (Mon Jul 28 07:25:25 2014) [sssd] [start_service] (0x0100): Queueing service ssh for startup (Mon Jul 28 07:25:25 2014) [sssd[pam]] [monitor_common_send_id] (0x0100): Sending ID: (pam,1) (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[pam]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [pam] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [monitor_common_send_id] (0x0100): Sending ID: (ssh,1) (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[ssh]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [ssh] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd[pac]] [monitor_common_send_id] (0x0100): Sending ID: (pac,1) (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[pac]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [pac] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd[nss]] [monitor_common_send_id] (0x0100): Sending ID: (nss,1) (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sss_names_init] (0x0100): Using re [(((?P<domain>[^\\]+)\\(?P<name>.+$))|((?P<name>[^@]+)@(?P<domain>.+$))|(^(?P<name>[^@\\]+)$))]. (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sbus_client_init] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_interop.example.com]. (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sss_dp_init] (0x0010): Failed to connect to monitor services. (Mon Jul 28 07:25:25 2014) [sssd[nss]] [sss_process_init] (0x0010): fatal error setting up backend connector (Mon Jul 28 07:25:25 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [nss] exited with code [3] (Mon Jul 28 07:25:25 2014) [sssd] [sbus_dispatch] (0x0080): Connection is not open for dispatching. (Mon Jul 28 07:25:26 2014) [sssd] [get_ping_config] (0x0100): Time between service pings for [interop.example.com]: [10] (Mon Jul 28 07:25:26 2014) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [interop.example.com]: [60] (Mon Jul 28 07:25:26 2014) [sssd] [start_service] (0x0100): Queueing service interop.example.com for startup /usr/libexec/sssd/sssd_be: error while loading shared libraries: libcares.so.2: cannot open shared object file: No such file or directory (Mon Jul 28 07:25:26 2014) [sssd] [mt_svc_exit_handler] (0x0040): Child [interop.example.com] exited with code [127] (Mon Jul 28 07:25:26 2014) [sssd] [mt_svc_exit_handler] (0x0010): Process [interop.example.com], definitely stopped!
(Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0040): Returned with: 1
(Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Terminating [ssh][10518] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Couldn't kill [ssh][10518]: [No such process] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Terminating [pac][10517] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Couldn't kill [pac][10517]: [No such process] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Terminating [nss][10516] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Couldn't kill [nss][10516]: [No such process] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Terminating [pam][10515] (Mon Jul 28 07:25:26 2014) [sssd] [monitor_quit] (0x0020): Couldn't kill [pam][10515]: [No such process]


The logs show show nothing useful but this problem started during the
ipa-client-install - the log shows:

   2014-07-23T18:40:22Z DEBUG args=/usr/sbin/authconfig --enablesssdauth
--enablemkhomedir --update --enablesssd
   2014-07-23T18:40:22Z DEBUG stdout=Starting oddjobd:        [  OK ]
   2014-07-23T18:40:22Z DEBUG stderr=
   2014-07-23T18:40:22Z INFO SSSD enabled
   2014-07-23T18:40:29Z DEBUG args=/sbin/service sssd restart
   2014-07-23T18:40:29Z DEBUG stdout=Stopping sssd: [FAILED]
   Starting sssd:                                [FAILED]

   2014-07-23T18:40:29Z DEBUG stderr=cat: /var/run/sssd.pid: No such file or
directory

   2014-07-23T18:40:29Z WARNING SSSD service restart was unsuccessful.
   2014-07-23T18:40:29Z DEBUG args=/sbin/chkconfig sssd on
   2014-07-23T18:40:29Z DEBUG stdout=

Any ideas? Have we seen this before? I suppose I could uninstall the ipa
client and re-install but I didn't want
to touch anything until I hear back.

Thanks!

-m

btw - All systems have been updated as of this evening. Kerberos works fine
but anything requiring
lookups is toast.





--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

Reply via email to