The 389ds service has crashed due to a pthread_mutex_lock. Environment: Ubuntu 16.04, FreeIPA 4.3, 389-Directory/1.3.4.9 B2016.109.158 Trigger condition: When Ambari operation concurrently adds five nodes, a new Hadoop keytab file will be added. The 389ds service with a large number of krb clients for authentication and adding service keytab to 389ds Replication will deadlock, while the 389ds service without krb client login is normal. Therefore, this is a pthread_mutex_lock issue of (krb client -> search 389ds) + (add krb service -> 389ds data replica). I understand that Berkeley Database (bdb) has many problems, according to this document: https://www.port389.org/docs/389ds/howto/howto-use-lmdb.html. But I don't understand why the freeipa 4.3 version is so fragile? My current environment is reproducible. Only when adding krb service keytab nodes in batches will it crash. How can I solve this problem in a lightweight way without upgrading the version or replacing bdb? My current solution is to kill ns-slapd and then start-dirsrv.
389ds crashed logs tailf access ``` [29/Apr/2026:14:31:23 +0800] conn=19 op=2735 SRCH base="dc=yydevops,dc=com" scope=2 filter="(&(|(objectClass=krbprincipalaux)(objectClass=krbprincipal)(objectClass=ipakrbprincipal))(|(ipaKrbPrincipalAlias=krbt... [29/Apr/2026:14:31:25 +0800] conn=67 op=6 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:25 +0800] conn=67 op=6 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:28 +0800] conn=67 op=7 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:28 +0800] conn=67 op=7 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:31 +0800] conn=67 op=9 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:31 +0800] conn=67 op=9 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:34 +0800] conn=67 op=10 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:34 +0800] conn=67 op=10 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:37 +0800] conn=67 op=12 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:37 +0800] conn=67 op=12 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:40 +0800] conn=67 op=13 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:40 +0800] conn=67 op=13 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:43 +0800] conn=67 op=15 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:43 +0800] conn=67 op=15 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:46 +0800] conn=67 op=16 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:46 +0800] conn=67 op=16 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:49 +0800] conn=67 op=18 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:49 +0800] conn=67 op=18 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:52 +0800] conn=67 op=19 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:52 +0800] conn=67 op=19 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:55 +0800] conn=67 op=21 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:55 +0800] conn=67 op=21 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:31:58 +0800] conn=67 op=22 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:31:58 +0800] conn=67 op=22 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:32:01 +0800] conn=67 op=24 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:32:01 +0800] conn=67 op=24 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:32:03 +0800] conn=69 fd=95 slot=95 connection from local to /var/run/slapd-YYDEVOPS-COM.socket [29/Apr/2026:14:32:03 +0800] conn=69 op=-1 fd=95 closed - B1 [29/Apr/2026:14:32:03 +0800] conn=70 fd=95 slot=95 connection from local to /var/run/slapd-YYDEVOPS-COM.socket [29/Apr/2026:14:32:03 +0800] conn=70 AUTOBIND dn="cn=Directory Manager" [29/Apr/2026:14:32:03 +0800] conn=70 op=0 BIND dn="cn=Directory Manager" method=sasl version=3 mech=EXTERNAL [29/Apr/2026:14:32:03 +0800] conn=70 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=Directory Manager" [29/Apr/2026:14:32:03 +0800] conn=70 op=1 SRCH base="cn=ipa-70-2.hiido.host.int.yy.com,cn=masters,cn=ipa,cn=etc,dc=yydevops,dc=com" scope=2 filter="(ipaConfigString=enabledService)" attrs="ipaConfigString cn" [29/Apr/2026:14:32:03 +0800] conn=70 op=1 RESULT err=0 tag=101 nentries=6 etime=0 notes=U [29/Apr/2026:14:32:03 +0800] conn=70 op=2 SRCH base="cn=schema" scope=0 filter="(objectClass=*)" attrs="attributeTypes objectClasses" [29/Apr/2026:14:32:03 +0800] conn=70 op=2 RESULT err=0 tag=101 nentries=1 etime=0 [29/Apr/2026:14:32:04 +0800] conn=67 op=25 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:32:04 +0800] conn=67 op=25 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:32:04 +0800] conn=70 op=3 UNBIND [29/Apr/2026:14:32:04 +0800] conn=70 op=3 fd=95 closed - U1 [29/Apr/2026:14:32:07 +0800] conn=67 op=27 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:32:07 +0800] conn=67 op=27 RESULT err=0 tag=120 nentries=0 etime=0 [29/Apr/2026:14:32:10 +0800] conn=67 op=28 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop" [29/Apr/2026:14:32:10 +0800] conn=67 op=28 RESULT err=0 tag=120 nentries=0 etime=0 ``` gdb_threads.log ``` ....... Thread 34 (Thread 0x7f91757f2700 (LWP 3627)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f9333e5384d in __db_pthread_mutex_lock () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #2 0x00007f9333efc66c in __lock_get_internal () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #3 0x00007f9333efd087 in __lock_get () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #4 0x00007f9333f284b7 in __db_lget () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #5 0x00007f9333e7082d in __bam_search () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #6 0x00007f9333e5b8bc in ?? () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #7 0x00007f9333e5d9bf in ?? () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #8 0x00007f9333f14ea5 in __dbc_iget () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #9 0x00007f9333f23f82 in __dbc_get_pp () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #10 0x00007f932fe520ce in idl_new_fetch () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #11 0x00007f932fe60713 in index_read_ext_allids () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #12 0x00007f932fe4ac3d in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #13 0x00007f932fe4b452 in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #14 0x00007f932fe4ba3a in filter_candidates_ext () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #15 0x00007f932fe4ca3f in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #16 0x00007f932fe4b9a2 in filter_candidates_ext () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #17 0x00007f932fe4ca3f in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #18 0x00007f932fe4b9a2 in filter_candidates_ext () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #19 0x00007f932fe4ca3f in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #20 0x00007f932fe4b9a2 in filter_candidates_ext () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #21 0x00007f932fe871a7 in subtree_candidates () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #22 0x00007f932fe88733 in ldbm_back_search () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #23 0x00007f933ef1dd90 in op_shared_search () from /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #24 0x0000562002c216df in ?? () #25 0x0000562002c1030f in ?? () #26 0x00007f933de7e088 in ?? () from /usr/lib/x86_64-linux-gnu/libnspr4.so #27 0x00007f933da256ba in start_thread (arg=0x7f91757f2700) at pthread_create.c:333 #28 0x00007f933d75b41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Thread 33 (Thread 0x7f9175ff3700 (LWP 3626)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f933de786a0 in PR_WaitCondVar () from /usr/lib/x86_64-linux-gnu/libnspr4.so #2 0x0000562002c0e09e in ?? () #3 0x0000562002c0f753 in ?? () #4 0x00007f933de7e088 in ?? () from /usr/lib/x86_64-linux-gnu/libnspr4.so #5 0x00007f933da256ba in start_thread (arg=0x7f9175ff3700) at pthread_create.c:333 #6 0x00007f933d75b41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Thread 32 (Thread 0x7f91767f4700 (LWP 3625)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f933de786a0 in PR_WaitCondVar () from /usr/lib/x86_64-linux-gnu/libnspr4.so #2 0x0000562002c0e09e in ?? () #3 0x0000562002c0f753 in ?? () #4 0x00007f933de7e088 in ?? () from /usr/lib/x86_64-linux-gnu/libnspr4.so #5 0x00007f933da256ba in start_thread (arg=0x7f91767f4700) at pthread_create.c:333 #6 0x00007f933d75b41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Thread 31 (Thread 0x7f9176ff5700 (LWP 3624)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f933de786a0 in PR_WaitCondVar () from /usr/lib/x86_64-linux-gnu/libnspr4.so #2 0x0000562002c0e09e in ?? () #3 0x0000562002c0f753 in ?? () #4 0x00007f933de7e088 in ?? () from /usr/lib/x86_64-linux-gnu/libnspr4.so #5 0x00007f933da256ba in start_thread (arg=0x7f9176ff5700) at pthread_create.c:333 #6 0x00007f933d75b41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Thread 30 (Thread 0x7f91777f6700 (LWP 3623)): #0 0x00007f933da2a730 in futex_wait (private=<optimized out>, expected=0, futex_word=0x7f92e9eaeb0c) at ../sysdeps/unix/sysv/linux/futex-internal.h:61 #1 futex_wait_simple (private=<optimized out>, expected=0, futex_word=0x7f92e9eaeb0c) at ../sysdeps/nptl/futex-internal.h:135 #2 __pthread_rwlock_wrlock_slow (rwlock=0x7f92e9eaeb00) at pthread_rwlock_wrlock.c:67 #3 0x00007f9333e536d7 in __db_pthread_mutex_lock () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #4 0x00007f9333f6abbb in __memp_fget () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #5 0x00007f9333e70307 in __bam_search () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #6 0x00007f9333e5b8bc in ?? () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #7 0x00007f9333e5fde5 in ?? () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #8 0x00007f9333f158a1 in __dbc_iput () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #9 0x00007f9333f10a25 in __db_put () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #10 0x00007f9333f26593 in __db_put_pp () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so #11 0x00007f932fe52f7f in idl_new_insert_key () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #12 0x00007f932fe62174 in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #13 0x00007f932fe62d04 in index_addordel_values_ext_sv () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #14 0x00007f932fe63161 in index_addordel_values_sv () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #15 0x00007f932fe63217 in index_addordel_entry () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #16 0x00007f932fe65f5a in ldbm_back_add () from /usr/lib/x86_64-linux-gnu/dirsrv/plugins/libback-ldbm.so #17 0x00007f933eecf6ba in ?? () from /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #18 0x00007f933eed0a38 in do_add () from /usr/lib/x86_64-linux-gnu/dirsrv/libslapd.so.0 #19 0x0000562002c103e8 in ?? () #20 0x00007f933de7e088 in ?? () from /usr/lib/x86_64-linux-gnu/libnspr4.so #21 0x00007f933da256ba in start_thread (arg=0x7f91777f6700) at pthread_create.c:333 #22 0x00007f933d75b41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Thread 29 (Thread 0x7f9177ff7700 (LWP 3622)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f933de786a0 in PR_WaitCondVar () from /usr/lib/x86_64-linux-gnu/libnspr4.so #2 0x0000562002c0e09e in ?? () #3 0x0000562002c0f753 in ?? () #4 0x00007f933de7e088 in ?? () from /usr/lib/x86_64-linux-gnu/libnspr4.so #5 0x00007f933da256ba in start_thread (arg=0x7f9177ff7700) at pthread_create.c:333 #6 0x00007f933d75b41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 ...... ``` -- _______________________________________________ FreeIPA-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://forge.fedoraproject.org/infra/tickets/issues/new
