On 02/23/2016 05:10 PM, Andy Thompson wrote:
On 02/23/2016 03:02 PM, Andy Thompson wrote:
Came across one of my replicas this morning with the following in
the error log
[20/Feb/2016:17:23:38 -0500] - libdb: BDB2055 Lock table is out of
available lock entries
[20/Feb/2016:17:23:38 -0500] entryrdn-index - _entryrdn_delete_key:
Deleting C1 failed; Cannot allocate memory(12)
[20/Feb/2016:17:23:38 -0500] - database index operation failed BAD
1031, err=12 Cannot allocate memory
[20/Feb/2016:17:23:38 -0500] -
index_del_entry(changenumber=1328662,cn=changelog, 0x26) failed
(12)
[20/Feb/2016:17:23:38 -0500] DSRetroclPlugin - delete_changerecord:
could not delete change record 1328662 (rc: 1)
[20/Feb/2016:17:23:38 -0500] - libdb: BDB2055 Lock table is out of
available lock entries
[20/Feb/2016:17:23:38 -0500] entryrdn-index - _entryrdn_get_elem:
Failed to position cursor at the key: 1328666: Cannot allocate
memory(12)
[20/Feb/2016:17:23:38 -0500] entryrdn-index - _entryrdn_delete_key:
Failed to position cursor at the key: 1328666: Cannot allocate
memory(12)
[20/Feb/2016:17:23:38 -0500] - libdb: BDB2055 Lock table is out of
available lock entries
[20/Feb/2016:17:23:38 -0500] - database index operation failed BAD
1031, err=12 Cannot allocate memory
[20/Feb/2016:17:23:38 -0500] -
index_del_entry(changenumber=1328663,cn=changelog, 0x26) failed
(12)
[20/Feb/2016:17:23:38 -0500] NSMMReplicationPlugin - changelog
program
- _cl5CompactDBs: failed to compact
5f1d2b12-cf1411e4-b055ba8a-f4b484f7; db error - 12 Cannot allocate
memory
[20/Feb/2016:17:23:38 -0500] DSRetroclPlugin - delete_changerecord:
could not delete change record 1328663 (rc: 1)
[20/Feb/2016:17:23:41 -0500] ldbm_back_delete - conn=0 op=0 [retry:
1]
No original_tombstone for changenumber=1330335,cn=changelog!!
And then nothing. Was troubleshooting some clients that were having
issues resolving some trusted domain users.
I restarted IPA and it rolled through a few thousand missing change
records
23/Feb/2016:08:39:34 -0500] DSRetroclPlugin - delete_changerecord:
could not delete change record 1328696 (rc: 32)
Any thoughts as to what might have caused the lock table errors?
in BerkeleyDB this means that the number of pages which would have to
be locked in one transaction exceeds the configured number of locks.
This could happen if eg a large group is deleted and for each member
of the group inside the same transaction the memberof attribute has
to be modified
Are there any configuration options to increase that setting? And would it
have caused the replica to become unresponsive?
you can change
nsslapd-db-locks
in the entry:
dn: cn=config,cn=ldbm database,cn=plugins,cn=config
yes. in that state it would not process updates, the txn should be finally
aborted and the system should recover,but ..
Is there any rule of thumb or anything I can look at to get an idea of what I
should increase that to or should it even be necessary?
The current setting has a default of 10000
cn=database,cn=monitor,cn=ldbm database,cn=plugins,cn=config
currently shows
nsslapd-db-current-locks: 82
What might cause that to spike up that significantly to deplete the locks?
That's a pretty huge jump.
I have given you an example of what operation could use a high number of
page locks, to find out what was going on in your case would require to
investigate which operations were active when the problem started, what
the entries modified added looked like ......
Running 389-ds-base-1.3.4.0-26.el7_2.x86_64
-andy
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
