[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
xenial verification: Reproducing the crash: ubuntu@xenial-sssd-1684295:~$ pidof sssd_be 3516 ubuntu@xenial-sssd-1684295:~$ id localuser (stuck for 30s or more) uid=1001(localuser) gid=1001(localuser) groups=1001(localuser) syslog shows the crash: Nov 13 13:05:55 xenial-sssd-1684295 systemd[1]: Starting Apport crash forwarding receiver... Nov 13 13:06:40 xenial-sssd-1684295 sssd: Killing service [LDAP], not responding to pings! Nov 13 13:06:54 xenial-sssd-1684295 sssd[be[LDAP]]: Starting up pid changed: ubuntu@xenial-sssd-1684295:~$ pidof sssd_be 4639 With the new package it works: Version table: *** 1.13.4-1ubuntu1.9 500 500 http://br.archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages ubuntu@xenial-sssd-1684295:~$ id localuser uid=1001(localuser) gid=1001(localuser) groups=1001(localuser),1(ldapusers) xenial verification succeeded. ** Tags removed: verification-needed-xenial ** Tags added: verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
@pam-s, as soon as you can confirm this patch fixes your problem (feel free to use my PPA packages), please let us know so we can proceed with the SRU. My test case reproduces the segfault, but I would like to be sure it also fixes it in your environment before continuing. Thanks again ** Description changed: [Impact] In this particular configuration, when ldap_rfc2307_fallback_to_local_users is set to true in /etc/sss/sssd.conf and a local user is a member of an ldap group and does not exist in the directory (other scenarios are possible), the sssd_be process segfaults and logins might be prevented. The original scenario is a bit more complex and involves setting up an Active Directory server, but with the help from the bug reporter (thanks @pam-s!) we managed to narrow it down to this simple test case. [Test Case] # Install the packages. When prompted, choose any password for the ldap admin $ sudo apt update; sudo apt install sssd slapd # create the sssd config $ sudo tee /etc/sssd/sssd.conf
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
** Description changed: [Impact] + In this particular configuration, when ldap_rfc2307_fallback_to_local_users is set to true in /etc/sss/sssd.conf and a local user is a member of an ldap group and does not exist in the directory (other scenarios are possible), the sssd_be process segfaults and logins might be prevented. - * An explanation of the effects of the bug on users and + The original scenario is a bit more complex and involves setting up an + Active Directory server, but with the help from the bug reporter (thanks + @pam-s!) we managed to narrow it down to this simple test case. - * justification for backporting the fix to the stable release. - - * In addition, it is helpful, but not required, to include an -explanation of how the upload fixes this bug. [Test Case] - * detailed instructions how to reproduce the bug + * detailed instructions how to reproduce the bug - * these should allow someone who is not familiar with the affected -package to reproduce the bug and verify that the updated package fixes -the problem. + * these should allow someone who is not familiar with the affected + package to reproduce the bug and verify that the updated package fixes + the problem. [Regression Potential] - * discussion of how regressions are most likely to manifest as a result + * discussion of how regressions are most likely to manifest as a result of this change. - * It is assumed that any SRU candidate patch is well-tested before -upload and has a low overall risk of regression, but it's important -to make the effort to think about what ''could'' happen in the -event of a regression. + * It is assumed that any SRU candidate patch is well-tested before + upload and has a low overall risk of regression, but it's important + to make the effort to think about what ''could'' happen in the + event of a regression. - * This both shows the SRU team that the risks have been considered, -and provides guidance to testers in regression-testing the SRU. + * This both shows the SRU team that the risks have been considered, + and provides guidance to testers in regression-testing the SRU. [Other Info] - - * Anything else you think is useful to include - * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board - * and address these questions in advance + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance ** Description changed: [Impact] In this particular configuration, when ldap_rfc2307_fallback_to_local_users is set to true in /etc/sss/sssd.conf and a local user is a member of an ldap group and does not exist in the directory (other scenarios are possible), the sssd_be process segfaults and logins might be prevented. The original scenario is a bit more complex and involves setting up an Active Directory server, but with the help from the bug reporter (thanks @pam-s!) we managed to narrow it down to this simple test case. - [Test Case] - * detailed instructions how to reproduce the bug + # Install the packages. When prompted, choose any password for the ldap admin + $ sudo apt update; sudo apt install sssd slapd - * these should allow someone who is not familiar with the affected - package to reproduce the bug and verify that the updated package fixes - the problem. + # create the sssd config + $ sudo tee /etc/sssd/sssd.conf
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
It's only xenial that is affected, that is, version 1.13.4 and perhaps earlier. Trusty, zesty and higher are OK. ** Description changed: - This is Ubuntu 16.04.2 LTS + [Impact] - sssd is configured to connect to two domains, our TPAD directory and - Active Directory. sssd starts up at boot time. As soon as I ssh login - (with any id, AD, TPAD or local), sssd fails with the error message in - the title. After that, we can only login with local ids, not TPAD or AD - ids. + * An explanation of the effects of the bug on users and - - Here is the output from systemctl status sssd after the failure: - root@dcmilphlum128:~# systemctl status sssd - â sssd.service - System Security Services Daemon -Loaded: loaded (/lib/systemd/system/sssd.service; enabled; vendor preset: enabled) -Active: failed (Result: exit-code) since Wed 2017-04-19 16:40:08 EDT; 7min ago - Process: 119143 ExecStart=/usr/sbin/sssd -D -f (code=exited, status=0/SUCCESS) - Main PID: 119145 (code=exited, status=1/FAILURE) + * justification for backporting the fix to the stable release. - Apr 19 16:39:47 dcmilphlum128.edc.nam.gm.com sssd[be[119187]: Starting up - Apr 19 16:39:51 dcmilphlum128.edc.nam.gm.com sssd[be[119191]: Starting up - Apr 19 16:39:57 dcmilphlum128.edc.nam.gm.com sssd[be[119206]: Starting up - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com sssd[119145]: Exiting the SSSD. Could not restart critical service [tpad]. - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com sssd[119149]: Shutting down - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com sssd[119148]: Shutting down - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com sssd[be[119146]: Shutting down - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com systemd[1]: sssd.service: Main process exited, code=exited, status=1/FAILURE - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com systemd[1]: sssd.service: Unit entered failed state. - Apr 19 16:40:08 dcmilphlum128.edc.nam.gm.com systemd[1]: sssd.service: Failed with result 'exit-code'. + * In addition, it is helpful, but not required, to include an +explanation of how the upload fixes this bug. - ** - Also, in kern.log I have four of these (I have retries set to 3): - Apr 19 16:39:59 dcmilphlum128 kernel: [ 6205.937807] sssd_be[12218]: segfault at 0 ip 7fecb32b6b94 sp 7ffce49a2230 error 4 in libsss_util.so[7fecb32a2000+6c000] - Apr 19 16:40:02 dcmilphlum128 kernel: [ 6206.980725] sssd_be[12253]: segfault at 0 ip 7f302de29b94 sp 7fffca943cc0 error 4 in libsss_util.so[7f302de15000+6c000] - Apr 19 16:40:05 dcmilphlum128 kernel: [ 6211.036205] sssd_be[12256]: segfault at 0 ip 7fd196169b94 sp 7ffd624249f0 error 4 in libsss_util.so[7fd196155000+6c000] - Apr 19 16:40:07 dcmilphlum128 kernel: [ 6225.081902] sssd_be[12257]: segfault at 0 ip 7fd1f669bb94 sp 7ffdd8e5bf80 error 4 in libsss_util.so[7fd1f6687000+6c000] + [Test Case] - *** - My sssd package are at 1.13.4: - sssd 1.13.4-1ubuntu1.1 amd64 - sssd-ad1.13.4-1ubuntu1.1 amd64 - sssd-ad-common 1.13.4-1ubuntu1.1 amd64 - sssd-common1.13.4-1ubuntu1.1 amd64 - sssd-ipa 1.13.4-1ubuntu1.1 amd64 - sssd-krb5 1.13.4-1ubuntu1.1 amd64 - sssd-krb5-common 1.13.4-1ubuntu1.1 amd64 - sssd-ldap 1.13.4-1ubuntu1.1 amd64 - sssd-proxy 1.13.4-1ubuntu1.1 amd64 - sssd-tools 1.13.4-1ubuntu1.1 amd64 + * detailed instructions how to reproduce the bug - *** - I upgraded all the sssd packages to 1.13.4-1ubuntu1.4 and had the same problem. + * these should allow someone who is not familiar with the affected +package to reproduce the bug and verify that the updated package fixes +the problem. - I downgraded them to 1.12.5-2 and was NOT able to reproduce the problem. + [Regression Potential] - I attached my sssd.conf file. + * discussion of how regressions are most likely to manifest as a result + of this change. + + * It is assumed that any SRU candidate patch is well-tested before +upload and has a low overall risk of regression, but it's important +to make the effort to think about what ''could'' happen in the +event of a regression. + + * This both shows the SRU team that the risks have been considered, +and provides guidance to testers in regression-testing the SRU. + + [Other Info] + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance -- You received
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
** Changed in: sssd (Ubuntu) Assignee: (unassigned) => Andreas Hasenack (ahasenack) ** Changed in: sssd (Ubuntu) Status: Triaged => In Progress -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
This PPA has my test packages: https://launchpad.net/~ahasenack/+archive/ubuntu/sssd-bad-initgroups- results-1684295/ -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
I got a small reproducer case. With a simple "id " command I get sssd_be to segfault, and with the above patch applied it no longer segfaults and also produces the correct result. I'll use that for the SRU test plan. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
I believe this is the upstream issue: https://pagure.io/SSSD/sssd/issue/3045 The patch is https://pagure.io/SSSD/sssd/c/5a0fb268e836e600d864ded7de5d935946ae6c61 ** Changed in: sssd (Ubuntu) Status: New => Triaged ** Changed in: sssd (Ubuntu) Importance: Undecided => High ** Tags added: bitesize server-next -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to sssd in Ubuntu. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
Thanks, I can track this down more easily now.
(gdb) bt
#0 sysdb_attrs_get_el_ext (attrs=attrs@entry=0x0,
name=name@entry=0x7f1e14bb504c "stamp", alloc=alloc@entry=true,
el=el@entry=0x7ffc041e48a8) at ../src/db/sysdb.c:326
#1 0x7f1e2d283d0d in sysdb_attrs_get_el (attrs=attrs@entry=0x0,
name=name@entry=0x7f1e14bb504c "stamp", el=el@entry=0x7ffc041e48a8) at
../src/db/sysdb.c:360
#2 0x7f1e14b6dda6 in sdap_attrs_get_sid_str
(mem_ctx=mem_ctx@entry=0x1664b40, idmap_ctx=0x1682ba0,
sysdb_attrs=sysdb_attrs@entry=0x0, sid_attr=0x7f1e14bb504c "stamp",
_sid_str=_sid_str@entry=0x7ffc041e4998) at
../src/providers/ldap/ldap_common.c:897
#3 0x7f1e14b7a878 in sdap_save_user (memctx=memctx@entry=0x1bc3c20,
opts=0x1679b20, dom=0x167aa80, attrs=0x0, _usn_value=_usn_value@entry=0x0,
now=now@entry=0)
at ../src/providers/ldap/sdap_async_users.c:160
#4 0x7f1e14b8be07 in sdap_get_initgr_user (subreq=0x0) at
../src/providers/ldap/sdap_async_initgroups.c:2896
#5 0x7f1e14b75428 in generic_ext_search_handler (subreq=0x0,
opts=) at ../src/providers/ldap/sdap_async.c:1668
#6 0x7f1e14b77908 in sdap_get_generic_op_finished (op=,
reply=, error=, pvt=) at
../src/providers/ldap/sdap_async.c:1561
#7 0x7f1e14b7638d in sdap_process_message (ev=,
sh=, msg=0x1664ae0) at ../src/providers/ldap/sdap_async.c:352
#8 sdap_process_result (ev=, pvt=) at
../src/providers/ldap/sdap_async.c:196
#9 0x7f1e2df90613 in ?? () from /usr/lib/x86_64-linux-gnu/libtevent.so.0
#10 0x7f1e2df8eb57 in ?? () from /usr/lib/x86_64-linux-gnu/libtevent.so.0
#11 0x7f1e2df8ad3d in _tevent_loop_once () from
/usr/lib/x86_64-linux-gnu/libtevent.so.0
#12 0x7f1e2df8aedb in tevent_common_loop_wait () from
/usr/lib/x86_64-linux-gnu/libtevent.so.0
#13 0x7f1e2df8eaf7 in ?? () from /usr/lib/x86_64-linux-gnu/libtevent.so.0
#14 0x7f1e2d2aff83 in server_loop (main_ctx=0x15ec060) at
../src/util/server.c:692
#15 0x00406412 in main (argc=8, argv=) at
../src/providers/data_provider_be.c:2994
(gdb) frame 0
#0 sysdb_attrs_get_el_ext (attrs=attrs@entry=0x0,
name=name@entry=0x7f1e14bb504c "stamp", alloc=alloc@entry=true,
el=el@entry=0x7ffc041e48a8) at ../src/db/sysdb.c:326
326 for (i = 0; i < attrs->num; i++) {
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to sssd in Ubuntu.
https://bugs.launchpad.net/bugs/1684295
Title:
sssd fails with 'Exiting the SSSD. Could not restart critical service
[tpad].
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions
--
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
I think the best way forward here is to get a core dump, so we can have a better idea of where the crash is happening. I induced a crash in my test sssd container, and since I have apport installed, a crash file was produced in /var/crash: # ll /var/crash/ total 644 drwxrwxrwt 2 root root 4 Nov 1 20:34 ./ drwxr-xr-x 13 root root 15 Sep 19 19:18 ../ -rwxr-xr-x 1 root root 0 Nov 1 20:34 .lock* -rw-r- 1 root root 593417 Nov 1 20:34 _usr_lib_x86_64-linux-gnu_sssd_sssd_be.0.crash Could you please check if you have a recent crash file related to sssd in that directory. If not, do this: sudo apt install apport # check the kernel core_pattern: # sysctl kernel.core_pattern kernel.core_pattern = |/usr/share/apport/apport %p %s %c %P And then restart sssd and induce the crash again, and then hopefully you will have a crash file and we can go from there. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to sssd in Ubuntu. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
Looking at this again. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to sssd in Ubuntu. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
Thanks for these -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to sssd in Ubuntu. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
Continuing on this bug is fine, thanks for getting back to us with a simplified configuration file. I reopened the bug so that it will be picked up again. ** Changed in: sssd (Ubuntu) Status: Expired => New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to sssd in Ubuntu. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1684295] Re: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad].
Thanks for filing this bug in Ubuntu. This one will be a bit complex for me to reproduce locally, as your sssd.conf file has a lot of customizations and multiple servers for each domain. It would be best if we could reduce this complexity as much as possible and still reproduce the problem. Have you tried with just one of your two domains to see if the crash still happens? For example, list just "gc" or just "tpad" in [sssd]->domains and then login with credentials from each. ** Changed in: sssd (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to sssd in Ubuntu. https://bugs.launchpad.net/bugs/1684295 Title: sssd fails with 'Exiting the SSSD. Could not restart critical service [tpad]. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1684295/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
