We are running openldap in cluster mode with MDB setup, and we started second cluster after some time and we observe that data is non synch between those 2 servers. So how do we synchronize the data.
> On Sep 7, 2018, at 8:00 AM, [email protected] wrote: > > Send openldap-technical mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://www.openldap.org/lists/mm/listinfo/openldap-technical > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openldap-technical digest..." > > > Send openldap-technical mailing list submissions to > [email protected] > When replying, please edit your Subject: header so it is more specific than > "Re: openldap-technical digest..." > > Today's Topics: > > 1. Replication issue? Data is different between master and > consumer with same entryCSNs (Dave Steiner) > 2. olcSecurity: tls=1 and olcLocalSSF= : what value should I > use? (Jean-Francois Malouin) > 3. Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I > use? (Quanah Gibson-Mount) > 4. Re: Replication issue? Data is different between master and > consumer with same entryCSNs (Frank Swasey) > 5. Re: Replication issue? Data is different between master and > consumer with same entryCSNs (Quanah Gibson-Mount) > 6. Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I > use? (Jean-Francois Malouin) > 7. Re: Replication issue? Data is different between master and > consumer with same entryCSNs (Dave Steiner) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 5 Sep 2018 16:49:44 -0400 > From: Dave Steiner <[email protected]> > To: [email protected] > Subject: Replication issue? Data is different between master and > consumer with same entryCSNs > Message-ID: <[email protected]> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > I've been noticing various data discrepancies between our LDAP master and > LDAP > consumers.? We are running OpenLDAP v2.4.44.? We have two masters running > "mirromode TRUE" and all updates go through a VIP that points to the first > one > unless it's not available (doesn't happen very often except for during > patches > and restarts). We have 13 consumers that replicate through that same VIP. > > Here's an example of our syncrepl for a client: > > syncrepl rid=221 > ? type=refreshAndPersist > ? schemachecking=on > ? provider="ldap://ldapmastervip.rutgers.edu/"; > ? bindmethod=sasl > ? saslmech=EXTERNAL > ? starttls=yes > ? tls_reqcert=demand > ? tls_protocol_min="3.1" > ? searchbase="dc=rutgers,dc=edu" > ? attrs="*,+" > ? retry="10 10 20 +" > ? logbase="cn=accesslog" > ? logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" > ? syncdata=accesslog > ? network-timeout=30 > ? keepalive=180:3:60 > > I check the contextCSN attributes on all the instances every day and they are > all in sync (except during any major changes, of course). But I occasionally > notice discrepancies in the data.... even though the contextCSNs and > entryCSNs > are the same.? For example (note hostnames have been changed): > > $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress > createTimestamp modifyTimestamp entryCSN > dn: uid=XXXX,ou=People,dc=rutgers,dc=edu > createTimestamp: 20121220100700Z > postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ 081021656 > entryCSN: 20180505002024.083133Z#000000#001#000000 > modifyTimestamp: 20180505002024Z > > $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX postalAddress > createTimestamp modifyTimestamp entryCSN > dn: uid=XXXX,ou=People,dc=rutgers,dc=edu > createTimestamp: 20121220100700Z > postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ 081021656 > entryCSN: 20180505002024.083133Z#000000#001#000000 > modifyTimestamp: 20180505002024Z > > So I'm trying to figure out why this happens (config issue, bug, ???) and > second, if I can't use the contextCSN to report that everything is fine, what > else can I do besides trying to compare ldif dumps. > > thanks, > ds > -- > Dave Steiner [email protected] > IdM, Enterprise Application Services ?? ASB101; 848.445.5433 > Rutgers University, Office of Information Technology > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://www.openldap.org/lists/openldap-technical/attachments/20180905/4e3872b4/attachment.html> > > ------------------------------ > > Message: 2 > Date: Thu, 6 Sep 2018 12:40:05 -0400 > From: Jean-Francois Malouin <[email protected]> > To: OpenLDAP Technical <[email protected]> > Subject: olcSecurity: tls=1 and olcLocalSSF= : what value should I > use? > Message-ID: <[email protected]> > Content-Type: text/plain; charset=us-ascii > > Sorry if this is long and naive, I'm making my way with OpenLDAP. > > I have this annoying problem of local access over ldapi:/// of a configured > mdb database using its rootDN. > > Some details: > > (I typically use ldapvi to access/modify/edit config as I'm an old wolf with > vi > hard-wired in my brain!) > > (Same could be done using native OpenLDAP utilities ldapadd/search/delete/etc: > just replace the ldapvi '-h' option with '-H' to specify the > protocol/host/port). > > Binding using EXTERNAL mech over local ldapi:/// works correctly for > 'cn=config'. > For example, here I made a mod to olcLogLevel for 'cn=config': > > ~# ldapvi -Y EXTERNAL -h ldapi:/// -b 'cn=config' > SASL/EXTERNAL authentication started > SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth > SASL SSF: 0 > 24 entries read > > add: 0, rename: 0, modify: 1, delete: 0 > Action? [yYqQvVebB*rsf+?] y > Done. > > Server logs for slapd show the binding with ssf=71: > > Sep 6 11:40:52 slapd[677]: conn=48667 fd=17 ACCEPT from > PATH=/var/run/slapd/ldapi (PATH=/var/run/slapd/ldapi) > Sep 6 11:40:52 slapd[677]: conn=48667 op=0 BIND dn="" method=163 > Sep 6 11:40:52 slapd[677]: conn=48667 op=0 BIND > authcid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" > authzid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" > Sep 6 11:40:52 slapd[677]: conn=48667 op=0 BIND > dn="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" mech=EXTERNAL > sasl_ssf=0 ssf=71 > > > However for the configured mdb database 'olcDatabase={1}mdb,cn=config' I have > set > > olcSecurity: tls=1 > > to force binding with StartTLS. Here the relevant config piece for it: > ('--out' makes ldapvi behave like ldapsearch). > > ~# ldapvi --out -Y EXTERNAL -h ldapi:/// -b 'olcDatabase={1}mdb,cn=config' > SASL/EXTERNAL authentication started > SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth > SASL SSF: 0 > > dn: olcDatabase={1}mdb,cn=config > objectClass: olcDatabaseConfig > objectClass: olcMdbConfig > olcDatabase: {1}mdb > olcDbDirectory: /var/lib/ldap > olcSuffix: dc=example,dc=com > olcRootDN: cn=admin,dc=example,dc=com > olcRootPW: {SSHA}XXXXXXXXXXXXXXXXXXXX > olcSecurity: tls=1 > ... > > However this setting prohibits me from binding to it using ldapi:/// with > EXTERNAL mech with its rootDN 'cn=admin,dc=example,dc=com' as I then get the > message: > > ~# ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -D 'cn=admin,dc=example,dc=com' > -b 'dc=example,dc=com' > > SASL/EXTERNAL authentication started > SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth > SASL SSF: 0 > Confidentiality required (13) > Additional information: TLS confidentiality required > > > I can however do a simple bind over StartTLS with the rootDN of the database > either over localhost or a remote client: > > ~# ldapsearch -LLL -Z -x -w xxxxxxxx -H ldap://localhost -D > 'cn=admin,dc=example,dc=com' -b 'dc=example,dc=com' > > slapd logs show: > > Sep 6 11:54:40 slapd[677]: conn=48699 fd=17 ACCEPT from IP=127.0.0.1:53542 > (IP=0.0.0.0:389) > Sep 6 11:54:40 slapd[677]: conn=48699 op=0 EXT oid=1.3.6.1.4.1.1466.20037 > Sep 6 11:54:40 slapd[677]: conn=48699 op=0 STARTTLS > Sep 6 11:54:40 slapd[677]: conn=48699 op=0 RESULT oid= err=0 text= > Sep 6 11:54:40 slapd[677]: conn=48699 fd=17 TLS established tls_ssf=256 > ssf=256 > Sep 6 11:54:40 slapd[677]: conn=48699 op=1 BIND > dn="cn=admin,dc=example,dc=com" method=128 > Sep 6 11:54:40 slapd[677]: conn=48699 op=1 BIND > dn="cn=admin,dc=example,dc=com" mech=SIMPLE ssf=0 > > So ssf=265... > > I guess I need to modify either 'olcSecurity: tls=1' in the database config or > add/insert the proper value for 'olcLocalSSF=' in the cn=config. What value > should I use in order to still force StartTLS over simple binding and allow > read/write/modify local access on the ldapi:/// listener. > > Regards! > jf > > > > ------------------------------ > > Message: 3 > Date: Thu, 06 Sep 2018 11:36:33 -0700 > From: Quanah Gibson-Mount <[email protected]> > To: Jean-Francois Malouin <[email protected]>, > OpenLDAP Technical <[email protected]> > Subject: Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I > use? > Message-ID: <4CC3F9658701FD268DFEC4C0@[192.168.1.10]> > Content-Type: text/plain; charset=us-ascii; format=flowed > > --On Thursday, September 06, 2018 1:40 PM -0400 Jean-Francois Malouin > <[email protected]> wrote: > >> I guess I need to modify either 'olcSecurity: tls=1' in the database >> config or add/insert the proper value for 'olcLocalSSF=' in the >> cn=config. What value should I use in order to still force StartTLS over >> simple binding and allow read/write/modify local access on the ldapi:/// >> listener. > > Hello, > > Just set: > > olcSecurity: ssf=1 > > that will allow either to work as *some* SSF level is then required. > > As long as you have tls=X, then it will always require TLS, regardless of > what the LocalSSF setting is configured to be. > > --Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> > > > > > ------------------------------ > > Message: 4 > Date: Thu, 6 Sep 2018 18:39:03 +0000 > From: Frank Swasey <[email protected]> > To: Dave Steiner <[email protected]> > Cc: "[email protected]" > <[email protected]> > Subject: Re: Replication issue? Data is different between master and > consumer with same entryCSNs > Message-ID: <[email protected]> > Content-Type: text/plain; charset="us-ascii" > > > >> On Sep 5, 2018, at 16:49, Dave Steiner <[email protected]> wrote: >> >> >> But I occasionally notice discrepancies in the data.... even though the >> contextCSNs and entryCSNs are the same. For example (note hostnames have >> been changed): >> >> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress >> createTimestamp modifyTimestamp entryCSN >> postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ 081021656 >> >> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX postalAddress >> createTimestamp modifyTimestamp entryCSN >> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu >> postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ 081021656 > > You do realize that as far as LDAP is concerned, those two postalAddress > values are identical. The matching rules for postalAddress are case > insensitive. > > Assuming that the mixed case one is what you prefer and the upper case one is > what you started with - one has to ask, how was it changed on the one server > and make certain that it was not done in a way that would not be sent through > the syncrepl process. > > - Frank > > > ------------------------------ > > Message: 5 > Date: Thu, 06 Sep 2018 11:48:06 -0700 > From: Quanah Gibson-Mount <[email protected]> > To: Dave Steiner <[email protected]>, > [email protected] > Subject: Re: Replication issue? Data is different between master and > consumer with same entryCSNs > Message-ID: <88F55B570F9FE65911DAA8EE@[192.168.1.10]> > Content-Type: text/plain; charset=us-ascii; format=flowed > > --On Wednesday, September 05, 2018 5:49 PM -0400 Dave Steiner > <[email protected]> wrote: > > Hi Dave, > > >> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress >> createTimestamp modifyTimestamp entryCSN >> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu >> createTimestamp: 20121220100700Z >> postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ >> 081021656 >> entryCSN: 20180505002024.083133Z#000000#001#000000 >> modifyTimestamp: 20180505002024Z >> >> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX >> postalAddress createTimestamp modifyTimestamp entryCSN >> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu >> createTimestamp: 20121220100700Z >> postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ >> 081021656 >> entryCSN: 20180505002024.083133Z#000000#001#000000 >> modifyTimestamp: 20180505002024Z > > Those entries appear identical to me. I assume you believe they are not > identical because the case doesn't match. However, postalAddress uses > caseIgnore normalization rules, so they are in fact identical as far as > LDAP is concerned. > > Also, given the entry was created in 2012 and there's no indication of when > the postalAddress attribute was modified, it's impossible to determine > when/how the divergence in capitalization occurred. You also don't provide > any information about the underlying database in use, which could be very > relevant, history wise. > > >> So I'm trying to figure out why this happens (config issue, bug, ???) and >> second, if I can't use the contextCSN to report that everything is fine, >> what else can I do besides trying to compare ldif dumps. > > So far, everything appears fine from a technical standpoint (the values > match per LDAP SYNTAX rules). > > I would suggest the following if you would like the case to match: > > dn: ... > changetype: modify > replace: postalAddress > postalAddress: ... > > That should correct the value on the providers and consumers to match case. > > Warm regards, > Quanah > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> > > > > > ------------------------------ > > Message: 6 > Date: Thu, 6 Sep 2018 15:59:54 -0400 > From: Jean-Francois Malouin <[email protected]> > To: Quanah Gibson-Mount <[email protected]> > Cc: OpenLDAP Technical <[email protected]> > Subject: Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I > use? > Message-ID: <[email protected]> > Content-Type: text/plain; charset=us-ascii > > Hi, > > * Quanah Gibson-Mount <[email protected]> [20180906 14:36]: >> --On Thursday, September 06, 2018 1:40 PM -0400 Jean-Francois >> Malouin <[email protected]> wrote: >> >>> I guess I need to modify either 'olcSecurity: tls=1' in the database >>> config or add/insert the proper value for 'olcLocalSSF=' in the >>> cn=config. What value should I use in order to still force StartTLS over >>> simple binding and allow read/write/modify local access on the ldapi:/// >>> listener. >> >> Hello, >> >> Just set: >> >> olcSecurity: ssf=1 >> >> that will allow either to work as *some* SSF level is then required. >> >> As long as you have tls=X, then it will always require TLS, >> regardless of what the LocalSSF setting is configured to be. > > Thank you for the pointer! > > jf > >> >> --Quanah >> >> >> -- >> >> Quanah Gibson-Mount >> Product Architect >> Symas Corporation >> Packaged, certified, and supported LDAP solutions powered by OpenLDAP: >> <http://www.symas.com> > > > > ------------------------------ > > Message: 7 > Date: Thu, 6 Sep 2018 16:43:49 -0400 > From: Dave Steiner <[email protected]> > To: Quanah Gibson-Mount <[email protected]>, Dave Steiner > <[email protected]>, [email protected] > Subject: Re: Replication issue? Data is different between master and > consumer with same entryCSNs > Message-ID: <[email protected]> > Content-Type: text/plain; charset=utf-8; format=flowed > > > > On 9/6/2018 2:48 PM, Quanah Gibson-Mount wrote: >> --On Wednesday, September 05, 2018 5:49 PM -0400 Dave Steiner >> <[email protected]> wrote: >> >> Hi Dave, >> >> >>> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress >>> createTimestamp modifyTimestamp entryCSN >>> ?dn: uid=XXXX,ou=People,dc=rutgers,dc=edu >>> ?createTimestamp: 20121220100700Z >>> ?postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ >>> 081021656 >>> ?entryCSN: 20180505002024.083133Z#000000#001#000000 >>> ?modifyTimestamp: 20180505002024Z >>> >>> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX >>> postalAddress createTimestamp modifyTimestamp entryCSN >>> ?dn: uid=XXXX,ou=People,dc=rutgers,dc=edu >>> ?createTimestamp: 20121220100700Z >>> ?postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ >>> 081021656 >>> ?entryCSN: 20180505002024.083133Z#000000#001#000000 >>> ?modifyTimestamp: 20180505002024Z >> >> Those entries appear identical to me. I assume you believe they are not >> identical because the case doesn't match.? However, postalAddress uses >> caseIgnore normalization rules, so they are in fact identical as far as LDAP >> is concerned. >> >> Also, given the entry was created in 2012 and there's no indication of when >> the postalAddress attribute was modified, it's impossible to determine >> when/how the divergence in capitalization occurred.? You also don't provide >> any information about the underlying database in use, which could be very >> relevant, history wise. > > Ok, bad example... Here's another example that's rather different: > > $ searchn -H ldap://ldapmaster.rutgers.edu uid=XXXX mail createTimestamp > modifyTimestamp entryCSN > dn: uid=XXXX,ou=People,dc=rutgers,dc=edu > createTimestamp: 20180504140010Z > mail: [email protected] > entryCSN: 20180831205041.549496Z#000000#001#000000 > modifyTimestamp: 20180831205041Z > > $ searchn -H ldap://ldapconsumer1.rutgers.edu uid=XXXX mail createTimestamp > modifyTimestamp entryCSN > dn: uid=XXXX,ou=People,dc=rutgers,dc=edu > createTimestamp: 20180504140010Z > mail: [email protected] > entryCSN: 20180831205041.549496Z#000000#001#000000 > modifyTimestamp: 20180831205041Z > > >> >> >>> So I'm trying to figure out why this happens (config issue, bug, ???) and >>> second, if I can't use the contextCSN to report that everything is fine, >>> what else can I do besides trying to compare ldif dumps. >> >> So far, everything appears fine from a technical standpoint (the values >> match >> per LDAP SYNTAX rules). >> >> I would suggest the following if you would like the case to match: >> >> dn: ... >> changetype: modify >> replace: postalAddress >> postalAddress: ... >> >> That should correct the value on the providers and consumers to match case. >> >> Warm regards, >> Quanah >> >> >> >> -- >> >> Quanah Gibson-Mount >> Product Architect >> Symas Corporation >> Packaged, certified, and supported LDAP solutions powered by OpenLDAP: >> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.symas.com&data=02%7C01%7Csteiner%40oit.rutgers.edu%7Ca13f8a75c55b4fdbdb6308d614294c60%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636718564940835797&sdata=pVe%2FLnPDf4Civ13qlr1pXPCb0icy%2BzTZUEAcnViaTCo%3D&reserved=0> >> >> >> > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > openldap-technical mailing list > [email protected] > http://www.openldap.org/lists/mm/listinfo/openldap-technical > > > ------------------------------ > > End of openldap-technical Digest, Vol 130, Issue 3 > **************************************************
