On 05/06/2015 11:10 AM, Łukasz Jaworski wrote:
Hi,

ipactl stops working after dirsrv-stop/start.

There are many changes in the changelog:
from 39399 to 44397

(…)
# 44393, changelog
dn: changenumber=44393,cn=changelog

# 44394, changelog
dn: changenumber=44394,cn=changelog

# 44395, changelog
dn: changenumber=44395,cn=changelog

# 44396, changelog
dn: changenumber=44396,cn=changelog

# 44397, changelog
dn: changenumber=44397,cn=changelog

# search result
search: 2
result: 11 Administrative limit exceeded

# numResponses: 5001
# numEntries: 5000


After some seconds dirsrv stops responding.

In error log:
[06/May/2015:11:00:04 +0200] 
agmt="cn=cloneAgreement1-replica09.local-pki-tomcat" (replica08:389) - Can't 
locate CSN 55100d8c0000069f0000 in the changelog (DB rc=-30988). If replication stops, 
the consumer may need to be reinitialized.
[06/May/2015:11:00:04 +0200] - ldbm_back_seq deadlock retry BAD 1601, err=0 
BDB0062 Successful return: 0

ldapsearch hangs. Dirsrv is not responding now.
if the  server is hanging, can you get a pstack

This replica is on virtual machine (ganeti). We had problems with replication 
to vm, but after force-sync all was fine. On physical servers all works fine.
the messages indicate there could be many concurrent operations, because individual ops are not fast enough, could your VM have less/slower resources than the physical machines ?

Lukasz Jaworski 'Ender'

Wiadomość napisana przez Ludwig Krispenz <lkris...@redhat.com> w dniu 6 maj 
2015, o godz. 10:52:

Hi,

there seem to be different issues,
- I don't know what the ipactl status is looking for when it generates the 
error message about no matching master,
but I don't think it is related to the retro changelog.

- the retro changelog errors for adding and deleting
-- the add failures are about aborted transactions because a page cannot be 
accessed, this maybe caused by concurrent mods on different backends, which 
want to update teh shared retro cl database.
the changenumber reprted seems to be increasing, one error is about 
changenumber 44975, the next about 45577, so it looks like changes into the 
changelog are written and teh changenumber increases
-- i'm not sure about the delete errors, but normally trimming would go on 
after such an error message, the changenumber attempted to delete are 
increasing.
Could you verify which changes are in the changelog, and if these are changing:
ldapsearch -b "cn=changelog" dn

On 05/06/2015 09:52 AM, Łukasz Jaworski wrote:
Hi,

One of our replica hanged up morning. Error log after dirsrv restart:
[06/May/2015:09:28:15 +0200] - Retry count exceeded in delete
[06/May/2015:09:28:15 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 38376 (rc: 51)
[06/May/2015:09:28:15 +0200] - Operation error fetching Null DN 
(6368aeb7-f3c111e4-ae70ce39-9b469c1f), error -30993.
[06/May/2015:09:28:15 +0200] - dn2entry_ext: Failed to get id for 
changenumber=44975,cn=changelog from entryrdn index (-30993)
[06/May/2015:09:28:15 +0200] - Operation error fetching 
changenumber=44975,cn=changelog (null), error -30993.
[06/May/2015:09:28:15 +0200] DSRetroclPlugin - replog: an error occured while 
adding change number 44975, dn = changenumber=44975,cn=changelog: Operations 
error.
[06/May/2015:09:28:15 +0200] retrocl-plugin - retrocl_postob: operation failure 
[1]
[06/May/2015:09:28:15 +0200] - ldbm_back_seq deadlock retry BAD 1601, err=0 
BDB0062 Successful return: 0
[06/May/2015:09:30:03 +0200] - ldbm_back_seq deadlock retry BAD 1601, err=0 
BDB0062 Successful return: 0
[06/May/2015:09:30:06 +0200] - Retry count exceeded in delete
[06/May/2015:09:30:06 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39297 (rc: 51)

I did "re-initialize" from other replica.

Now ipactl doesn't work. Shows: Configured hostname 'replica09.local' does not 
match any master server in LDAP. On lists replica09 is exists (twice)

# ipactl status
Failed to get list of services to probe status!
Configured hostname 'replica09.local' does not match any master server in LDAP:
replica01.local
replica02.local
replica03.local
replica04.local
replica05.local
replica06.local
replica07.local
replica08.local
replica09.local
replica10.local
replica09.local

After dirsrv stop/start:

In error logs there are many:
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39290 (rc: 32)
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39291 (rc: 32)
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39292 (rc: 32)
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39293 (rc: 32)
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39294 (rc: 32)
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39295 (rc: 32)
[06/May/2015:09:50:30 +0200] DSRetroclPlugin - delete_changerecord: could not 
delete change record 39296 (rc: 32)
etc.

[06/May/2015:09:51:08 +0200] - Operation error fetching Null DN 
(9f51430a-f3c411e4-927ece39-9b469c1f), error -30993.
[06/May/2015:09:51:08 +0200] - dn2entry_ext: Failed to get id for 
changenumber=45577,cn=changelog from entryrdn index (-30993)
[06/May/2015:09:51:08 +0200] - Operation error fetching 
changenumber=45577,cn=changelog (null), error -30993.
[06/May/2015:09:51:08 +0200] DSRetroclPlugin - replog: an error occured while 
adding change number 45577, dn = changenumber=45577,cn=changelog: Operations 
error.
[06/May/2015:09:51:08 +0200] retrocl-plugin - retrocl_postob: operation failure 
[1]
[06/May/2015:09:51:08 +0200] - ldbm_back_seq deadlock retry BAD 1601, err=0 
BDB0062 Successful return: 0

Packages:
freeipa-server-4.1.3-2.fc21.x86_64
389-ds-base-1.3.3.8-1.fc21.x86_64
389-ds-base-libs-1.3.3.8-1.fc21.x86_64

Best regards,
Ender

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to