Re: dynlist vs memberof performance issues

[Mark Cairney]

Hi both,

Just to let you know that with my dataset I see a consistently slow
(10-20 secs) lookup of a user's memberOf group membership using dynamic
dynlist overlays on 2.5 and 2.6 compared with the old "memberof" overlay.

My dynlist config is as below:

dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig

It's fairly common to use the memberOf attribute to look up the groups a
particular user is a member of e.g. for access control purposes. We use
it routinely for this purpose on our Shibboleth IdPs which use OpenLDAP
as a datasource.

In terms of data size/complexity we've currently got around 500K users
(each with their own unique group in a separate OU) with the bulk of our
group membership managed by Grouper. We've got about 100K groups in
there(!) ranging in size from a single "placeholder" member up to about
200K members for our applicant/alumni groups.

The total DIT size (dumped via slapcat) is just under 2GB and our
(freshly slapcat'ed and slapadd'ed mdb database size is currently
sitting at 5.1GB). We're running on Centos 7 boxes with 12vCPUs and 64GB
RAM each which I think should be enough CPU/memory resource for our usage?

On a related note for a production environment should we be targeting
the 2.5 or 2.6 branch? I'd like to upgrade our production servers off
2.4 before the start of next term.


Kind regards,

Mark


On 26/07/2022 00:16, Paul B. Henson wrote:

This email was sent to you by someone outside the University.
You should only click on links or attachments if you are certain that
the email is genuine and the content is safe.

On 7/25/2022 10:38 AM, Shawn McKinney wrote:


As you (and others) have pointed out, there's a significant
performance penalty for searching attributes generated by dylist.


I'm still seeing performance issues with queries that simply return
memberOf, with no reference to it in the actual search filter.

For example, this query which searches on the static uid attribute and
returns memberOf:

time ldapsearch -H ldapi:/// uid=henson memberOf

Most of the time it completes in fractions of a second:

real0m0.187s
user0m0.005s
sys 0m0.003s

But sometimes it takes 5 seconds, 10 seconds, or even more. These
extremely slow response times coordinate with a high read I/O percentage
on the server and the high number of page faults on the slapd process.

When I first deployed 2.5, sometimes the server would get into a state
where every query that requested memberOf would take in excess of 30
seconds to return until the server was restarted. I cranked up the
memory on the servers and at this point I have had no more reoccurrences
of that behavior, but I am still regularly seeing occasional slow
performance on the queries and high read I/O percentages.

The servers have way more memory now than they should need to fully
cache the entire database:

# du -sh /var/symas/openldap-data-cpp
2.6G/var/symas/openldap-data-cpp

# free -m
  totalusedfree  shared buff/cache
available
Mem:   4818 703 124   0 3991
   3831
Swap:  2047 1361911

I haven't been able to correlate the slow response times with any other
external criteria such as updates or query load. Sometimes it's just
slow 8-/. We never saw this problem under 2.4 which used the previous
implementation of dynlist to generate memberOf. I definitely appreciate
the ability to query on dynamic memberOf that was added in 2.5, but it
would be nice to sort out this performance issue.


--
/********

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

***/

The University of Edinburgh is a charitable body, registered in Scotland, with 
registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh 
Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

dynlist vs memberof performance issues

[Mark Cairney]

x: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig

dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig

dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig

dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig


--
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

***/


The University of Edinburgh is a charitable body, registered in Scotland, with 
registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh 
Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

Re: OpenLDAP 2.5 plans and community engagement

[Mark Cairney]

On 30/07/2019 15:42, Michael Ströder wrote:
> On 7/30/19 11:20 AM, Ulrich Windl wrote:
>> Don't get me wrong: We can make it big (CPUs, RAM, Disks, energy consumption
>> ,cooling requirement), but isn't "making it small" more of an art? Today's
>> software mostly isn't "using a lot of memory" but rather "wasting a lot of
>> memory" IMHO.
> 
> lmdb's memory and disk footprint is small. My Æ-DIR development VMs are
> really small (~200 MB RAM) and there are various web components running
> on the providers.
> 
> I even tested this stuff with Raspberry PI model 1.
> And it did not consume too much resources.
> (Of course SD cards have really slow disk I/O.)
> 
> AFAICS there is only one case where back-mdb is significantly slower
> than back-hdb: ITS#8875. But this is actively worked on.

*cough* I think you mean ITS#7657 ;-)
It's good to see work being done on this issue and I can report that the
performance is massively improved in the current 2.4.48 RC release
compared to 2.4.47.

We actually managed to move away from using aliases due to this issue
and another issue (this time on HDB) where the whole system ground to a
halt when you went over 64K alias entries.
(See my email to the list on 16th Nov 2015- I thought I'd submitted an
ITS but I can't find it ao maybe I didn't)

> 
> So stop spreading FUD about lmdb. If you provide real-world evidence
> that back-mdb consumes more resources than back-hdb then present
> seriously worked out test results.
> 
> Ciao, Michael.
> 

With the exception of the issue above I've found back-mdb to require
less maintenance and be less prone to deadlocking under load and
corruption than back-bdb/hdb. Having the database backend built-in and
updated also means one less package dependency to worry about ;-)


RE: the community engagement time permitting I'd be happy to look over/
help out with the web documentation. One thing that might be useful is
some form of "cookbook" with recipes for setting up some common OpenLDAP
configs (e.g. single node, 2-way MMR, n-way MMR, single-master with
multiple replicas etc).

Kind regards,

Mark

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Replication issues with delta-syncrepl, MMR and memberOf overlay on 2.4.47

[Mark Cairney]

On 06/06/2019 16:30, Quanah Gibson-Mount wrote:
> Hi Mark,
> 
> --On Thursday, June 06, 2019 1:12 PM +0100 Mark Cairney
>  wrote:
> 
>>> I.e., the only way you can ensure slapo-memberof will be "ok" in an
>>> replicated environment (syncrepl or delta-syncrepl) is if you can
>>> guarantee a REFRESH will never occur.
>>
>> I think this was done in part to mitigate the behaviour in that ITS.
>> If memberOf isn't compatible with syncrepl then does that mean you can't
>> have replication and use the memberOf overlay? This would pose a problem
>> for many medium-large installations surely?
>> Looking back through the mailing list archive I do note a discussion of
>> the whys and wherefores in September 2018.
> 
> Right, the primary issue is that if a server goes into REFRESH mode, the
> order in which the entries are sent back may not allow the
> slapo-memberOf overlay to rebuild the groups correctly.
> 
> 
>> The man page suggests using the dynlist overlay instead but given my
>> limited understanding I don't see how that would work given the most
>> common use case of the memberOf overlay is to get the group memberships
>> when the user object is queried e.g.
>> ldapsearch "uid=mcairney" "*" memberof
> 
> I gave some examples in
> <https://www.openldap.org/its/index.cgi/?findid=8613>.  But none of it
> is pretty.

Ahh OK it looks like you're editing the "memberof" attribute in the user
(without the overlay) which is then used by the dynlist overlay to
generate group memberships on the fly.

Clever but I don't think we could switch over to this approach anytime
soon as the majority of our groups are automatically maintained by an
external application called "Grouper". It could potentially be
configured to do something along these lines but it wouldn't be
straightforward!

Something to think about/experiment with though

> 
>> In general though it does seem like replication and attributes
>> maintained locally via overlays (memberOf, ppolicy etc) are an absolute
>> minefield!
> 
> Yes, it is.  Unfortunately, Microsoft has never defined how memberOf
> should work in a replicated environment, and the RFC for ppolicy doesn't
> either. At least with ppolicy, one can configure back-ldap/slapo-chain
> to forward from the updates from a replica back to a provider so they
> get replicated out.
> 
>>> As I've noted a number of times on the list, overlay instantiation order
>>> is important for operation interception/processing.  The syncprov
>>> overlay should be the first instantiated overlay, followed by accesslog,
>>> in a delta-syncrepl setup.  In the above, this is clearly not the case.
>>
>> I thought I'd seen this mentioned but wasn't 100% sure. I've now
>> re-ordered my overlays as follows:
> 
> Great!
> 
>>> I would additionally note that the syncprov overlay is missing a
>>> sessionlog setting, where the default is likely much smaller than
>>> required for mitigating ITS#8125.
>>
>>
>> ...and I've set olcSpSessionlog to be 500 (I'm not sure what the default
>> is- 100?) Hopefully that's sufficient to avoid ITS#8125
> 
> It needs to be a value larger than the total number of entries in your
> database.

Thanks- I've now set it 200 based on our present entry count of
~1.25M. We've got 64GB RAM on these boxes so hopefully with a 30GB max
DB size set we should have enough.

Many thanks,

Mark

> 
> Regards,
> Quanah
> 
> 
> 
> -- 
> 
> Quanah Gibson-Mount
> Product Architect
> Symas Corporation
> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
> <http://www.symas.com>
> 
> 

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Replication issues with delta-syncrepl, MMR and memberOf overlay on 2.4.47

[Mark Cairney]

Hi Quanah,


On 05/06/2019 19:09, Quanah Gibson-Mount wrote:
> --On Wednesday, June 05, 2019 11:38 AM +0100 Mark Cairney
>  wrote:
> 
>> Hi,
>>
>> We currently run a multi-master setup where all 4 of our servers
>> replicate from each other via delta-syncrepl but all our writes are
>> directed at a selected "master" server.
>>
>> I've noticed recently that we are suffering sync issues and this
>> coincides with us upgrading from 2.4.46 with a patch for ITS8843 to
>> 2.4.47.
> 
> Hi Mark,
> 
> Error 0x14 would be attribute type or value already exists.  This would
> indicate a fundamental problem of there being discrepencies between your
> servers.  I.e., your databases are out of sync.

That smells a little bit like NTP sync but these boxes are all set to
aggressively poll a single NTP server:

server 129.215.205.191  minpoll 4 maxpoll 6
restrict 129.215.205.191 nomodify notrap




> 
>> There is a (possibly self-inflicted) point of pain where we have an
>> exattrs=memberOf in our syncrepl config to work around another
>> replication issue so this means that any user objects which are
>> REFRESH'ed end up losing all their group memberships.
> 
> You are aware the slapo-memberof(5) man page specifically states it is
> not compatible with syncrepl based replication, in particular because of
> the way in which the REFRESH phase functions combined with user/group
> entry location (creation time) in the database?
> 
> I would add that the exattrs line shouldn't be necessary with a proper
> configuration.  I'm not sure what issue you were trying to avoid with
> this setting.  The ITS#8444 regression test in the testsuite
> specifically has a 4-way MMR setup with memberof, and requires no such
> setting nor does it exhibit the issues you mention.
> 
> I.e., the only way you can ensure slapo-memberof will be "ok" in an
> replicated environment (syncrepl or delta-syncrepl) is if you can
> guarantee a REFRESH will never occur.

I think this was done in part to mitigate the behaviour in that ITS.
If memberOf isn't compatible with syncrepl then does that mean you can't
have replication and use the memberOf overlay? This would pose a problem
for many medium-large installations surely?
Looking back through the mailing list archive I do note a discussion of
the whys and wherefores in September 2018.

The man page suggests using the dynlist overlay instead but given my
limited understanding I don't see how that would work given the most
common use case of the memberOf overlay is to get the group memberships
when the user object is queried e.g.
ldapsearch "uid=mcairney" "*" memberof

In general though it does seem like replication and attributes
maintained locally via overlays (memberOf, ppolicy etc) are an absolute
minefield!

> 
> 
> As for the configuration of the server, see below.
> 
>> Are there any known bugs/ regressions with delta-syncrepl in 2.4.47?
>> One idea I've had is to go to a true single-master config by setting the
>> current consumers to be read-only and having a single olcSyncrepl clause
>> for the master on these 3 servers.
>>
>> dn: olcOverlay={0}dynlist,olcDatabase={1}mdb,cn=config
>> objectClass: olcOverlayConfig
>> objectClass: olcDynamicList
>> olcOverlay: {0}dynlist
>> olcDlAttrSet: {0}groupOfURLs memberURL
>>
>> dn: olcOverlay={1}memberof,olcDatabase={1}mdb,cn=config
>> objectClass: olcOverlayConfig
>> objectClass: olcMemberOf
>> olcOverlay: {1}memberof
>> olcMemberOfDangling: ignore
>> olcMemberOfRefInt: TRUE
>> olcMemberOfGroupOC: groupOfNames
>> olcMemberOfMemberAD: member
>> olcMemberOfMemberOfAD: memberOf
>>
>> dn: olcOverlay={2}syncprov,olcDatabase={1}mdb,cn=config
>> objectClass: olcOverlayConfig
>> objectClass: olcConfig
>> objectClass: top
>> objectClass: olcSyncProvConfig
>> olcOverlay: {2}syncprov
>>
>> dn: olcOverlay={3}accesslog,olcDatabase={1}mdb,cn=config
>> objectClass: olcOverlayConfig
>> objectClass: olcAccessLogConfig
>> olcOverlay: {3}accesslog
>> olcAccessLogDB: cn=accesslog
>> olcAccessLogOps: writes
>> olcAccessLogPurge: 02+00:00 00+04:00
>> olcAccessLogSuccess: TRUE
> 
> 
> As I've noted a number of times on the list, overlay instantiation order
> is important for operation interception/processing.  The syncprov
> overlay should be the first instantiated overlay, followed by accesslog,
> in a delta-syncrepl setup.  In the above, this is clearly not the case.

I thought I'd seen this mentioned but wasn't 100% sure. I've now
re-ordered my overlays as follows:

dn: olcDatabase={1}mdb,cn=config

# {0}syncprov, {1}mdb, config
dn: olcOverlay={0

Replication issues with delta-syncrepl, MMR and memberOf overlay on 2.4.47

[Mark Cairney]

jectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpNoPresent: TRUE
olcSpReloadHint: TRUE

dn: olcDatabase={3}monitor,cn=config
objectClass: olcMonitorConfig
objectClass: olcDatabaseConfig
objectClass: olcConfig
objectClass: top
olcDatabase: {3}monitor
olcAccess: {0}to dn.subtree="cn=monitor" by
dn.exact="cn=Manager,cn=config" write by * none

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Quick question about OpenLDAP Server CA certificate handling

[Mark Cairney]

Hi Andreas,

Thanks- the discussion in the ITS is very useful/interesting. I should
clarify that we're referring to a commercially-signed certificate for
our LDAP server so clients should already have the root CA in their
trust store.

We also aren't currently doing client cert verification and tend to use
service accounts for client authentication. We may look to support
client cert verification in the future using our own internal CA.

Kind regards,
Mark


On 13/04/2019 15:48, A. Schulze wrote:
> 
> 
> Am 11.04.19 um 13:35 schrieb Mark Cairney:
> 
> Hello Mark,
> 
>> However based on our understanding of how SSL works we should only
>> actually need the intermediate(s) in there as the client should have the
>> root and then compare the intermediate provided by the server and only
>> trust it if it can use this in conjunction with it's copy of the root
>> certificate to complete the chain of trust.
>>
>> Based on this we configure our web servers to only have the
>> intermediate(s) in their chain (and in fact SSL Labs marks you down if
>> you have the root in there too).
> That's best practice for *any* TLS server.
> 
> have a look at https://www.openldap.org/its/index.cgi?findid=8586
> With the referenced patch I can setup
>  TLSCertificateFile /path/to/cert+intermediate.pem
>  TLSCertificateKeyFile /path/to/privkey.pem
> 
> I have no TLSCACertificateFile at all because I don't use certificates
> to authenticate ldap clients...
> 
>> Of course we do realise LDAP is not HTTP!
> I think, it *is* very similar...
> 
> Andreas
> 
> 

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Quick question about OpenLDAP Server CA certificate handling

[Mark Cairney]

Hi,

Having just updated our SSL certificates on our OpenLDAP server led us
to review the contents of our "bundle" file referenced in
"olcTLSCACertificateFile".

According to the documentation at:
https://www.openldap.org/doc/admin24/tls.html it states "This directive
specifies the PEM-format file containing certificates for the CA's that
slapd will trust. The certificate for the CA that signed the server
certificate must be included among these certificates. If the signing CA
was not a top-level (root) CA, certificates for the entire sequence of
CA's from the signing CA to the top-level CA should be present. Multiple
certificates are simply appended to the file; the order is not significant."

However based on our understanding of how SSL works we should only
actually need the intermediate(s) in there as the client should have the
root and then compare the intermediate provided by the server and only
trust it if it can use this in conjunction with it's copy of the root
certificate to complete the chain of trust.

Based on this we configure our web servers to only have the
intermediate(s) in their chain (and in fact SSL Labs marks you down if
you have the root in there too).

Of course we do realise LDAP is not HTTP!

We're running OpenLDAP 2.4.47 linked against OpenSSL on Scientific Linux
7.5.

Kind regards,
Mark

-- 
/****

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: slapd: null_callback : error code 0x14

[Mark Cairney]

Thanks to Ondrej for persisting with getting to the bottom of this
rather annoying bug :-)

Unfortunately I won't be at LDAPCon this year otherwise I'd buy you a beer!

On 25/09/2017 15:31, Ondřej Kuzník wrote:
> On Fri, Sep 22, 2017 at 09:15:56PM -0700, Paul B. Henson wrote:
>> On Fri, Sep 22, 2017 at 08:50:38AM -0700, Quanah Gibson-Mount wrote:
>>> Oh, I thought you had said you only had two masters.  This could well be 
>>
>> Ah, my bad, there are a total of 4 nodes, and while technically I guess
>> they could all be "masters", only two of them ever receive writes, one
>> is the primary behind a hardware load balancer and the other is the
>> secondary; so in my head I have two masters and two read only systems.
>> Which I suppose isn't really accurate from an openldap architecture
>> perspective, sorry.
>>
>>> ITS#8444 (ignore the ITS title, it has nothing to do with memberOf), where 
>>> there are out of sync problems with 3+ MMR nodes and delta-syncrepl when 
>>> syncprov checkpoints.
>>
>> Oh, I remember that ITS, I thought I'd fixed that issue by getting rid
>> of the memberOf overlay and switching to dynlist 8-/. It seems since I
>> stopped paying attention to it it's moved on in other directions.
>>
>> I see there was a proposed patch posted on 8/25 that's been applied to
>> RE24, I'll add that to my system and see if the issue goes away. Am I
>> correct in my assumption that the patch only needs to be applied to the
>> system that is receiving the updates?
> 
> I'd apply it everywhere you have syncprov configured, these could send a
> cookie with too little information for a replica to spot and skip a
> duplicate.
> 

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: ppolicy overlay and MMR experiencing frequent delta-sync lost issue

[Mark Cairney]

On 12/01/2017 22:27, Michael Ströder wrote:

You could try using option exattrs with syncrepl statement, see slapd.conf(5).

Ciao, Michael.


Awesome, sounds exactly what I was looking for!

Kind regards,

Mark

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: ppolicy overlay and MMR experiencing frequent delta-sync lost issue

[Mark Cairney]

Hi Quanah,

On 12/01/17 16:06, Quanah Gibson-Mount wrote:


The correct fix is to modify your syncrepl configuration so that those
attributes are ignored by the syncrepl client.  There is no patch to the
code necessary.



Possibly a dumb question but do you have a worked example of this? The 
usual "get-all" stanza for this would "*, +" and as far as I'm aware you 
can't subtract attributes from the list returned i.e. search for all 
attributes *except* pwdFailureTime.
Does this mean you would need to list all the operational attributes you 
do want replicated (and isn't there a risk that you could break things 
if you were to miss out the wrong ones).


My thinking is that this would be a suitable workaround with the issue 
I've been experiencing with the memberOf attribute- if it isn't picked 
up by syncrepl then each server will (correctly) maintain it's own 
memberOf attributes individually.



Regards,
Quanah

--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:






--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: How to move from hdb to mdb

[Mark Cairney]

On 22/09/16 02:21, Dan White wrote:

> 
> Consider converting in two steps - converting your database to mdb first,
> then converting to slapd-config.
> 
> 

I did things the opposite way round:

1. slapcat your data database
2. slapcat your config database
3. Modify the generated config database ldif to correspond with that of
an mdb database i.e. change the objectclass, remove any bdb-specific
configuration, add the mdb specifics (MaxSize, maxReaders etc- there's a
lot less tunables in mdb)
3. Stop slapd and move your "old" config aside
4. slapadd your new config database
5. If 4 is successful, move your data directory aside and
6. Slapadd your data ldif.
7. Go for a (short) cup of tea
8. Fire up slapd.
-- 
/********

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: openldap 2.4.44 + ITS 8432 patch still infinitely replicates

[Mark Cairney]

Hi,

I've been away holidaying in Ireland  and I suspect I'll have a busy 
week or two catching up with things and prepping/ upgrading systems for 
the new academic year.


The systems I was using to test this are our old Live servers and are 
thus due to be decomissioned soon but if any further tests are required 
by me to progress things let me know.


Kind regards,

Mark

On 29/07/2016 22:30, Paul B. Henson wrote:

On Thu, Jul 28, 2016 at 04:09:27PM -0700, Quanah Gibson-Mount wrote:


The bug noted in ITS#8460 is not present in 2.4 at all. Quanah is also
running experimental backports of features from 2.5 and forgets to
mention that sometimes. This particular issue is from 2.5 (and is now
fixed in git).
Most likely it's the same issue as #8448. You can disregard both of these.

Ah, thanks for the clarification Howard :).


Same as ITS8462 as well.  So all 3 of those are off the table.  Other than
those, I haven't had problems with 2.4.44 + ITS8432. ;)

Cool. Are you using the memberOf overlay? It doesn't look like there's
been any update on ITS 8444 that Mark Cairney reported a month or so ago
in which he said he was seeing replication issues with 2.4.44 + ITS8432
fix for adds/deletes of objects managed by the overlay.

Mark, any progress on resolving that issue?

Thanks everybody...




--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: RHEL7 issues maybe???

[Mark Cairney]

Hi Frank,


On 01/07/16 14:15, Frank Swasey wrote:
> Today at 7:15am, Mark Cairney wrote:
> 
>> Another gotcha is w.r.t. rate limiting as both journald and rsyslog
>> implement this independently of one another. Disabling this completely
>> can make the bottleneck mentioned above even more apparent! Setting your
>> OpenLDAP logging level appropriately can mitigate this (I log at Stats+
>> Sync)
> 
> I am also running with
> 
> loglevel stats sync
> 
> and my servers are busy enough that journald misses a lot of the
> messages even with ryslogd's rate limiting turned off.  Since Red Hat's
> advice is to *not* remove the rate limiting in journald, I've not found
> a sweet spot yet where the sync logging does not get limited when the
> empty consumer is drinking from the fire hose.  ... any experience you
> can help me with would be greatly appreciated.
> 

My present /etc/systemd/journald.conf contains the following- these are
the defaults from our config management system and I haven't found a
need to adjust them yet (although they do look a bit looser than the RH
defaults):

[Journal]
Storage=persistent
SplitMode=login
RateLimitInterval=10s
RateLimitBurst=3000

My corresponding rsyslog settings are:
# Standard preamble

$ModLoad imuxsock
$ModLoad imjournal
$ModLoad immark

# Tuning
$OmitLocalLogging on
$IMJournalStateFile imjournal.state

$SystemLogRateLimitInterval 0

# Discard audit log entries
user.info ~

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Configure imjournal rate limiting
$imjournalRatelimitInterval 60
$imjournalRatelimitBurst 10

# CAuth logs
local4.* -/usr/local/authz/var/log/openldap.log



For day-to-day operations I'm not noticing any limiting kicking in.
During operations that generate a lot of logs e.g. re-syncing a replica,
SLAMD benchmarking logs do indeed get lost but this is preferable to the
alternative where the logger becomes the bottleneck and slapd is
essentially sitting there waiting for logs to be written before continuing.

RE: bringing up a replica I'd avoid using syncrepl from scratch- I find
building the slave from a dump of the master is far quicker and more
reliable. I tend to do (as a brief/trivial example):
(master)
slapcat -n 1 -o ldif-wrap=no  -l /tmp/master

scp it across to the slave
(slave)
rm -f /var/openldap-data/database/*.mdb
rm -f /var/openldap-data/accesslog/*.mdb
slapadd -q -w -n1 -l /tmp/master.ldif

Importing this with slapadd takes me about 5 mins for a 1.5G ldif file
using mdb- it took about 15-20 mins when I was using bdb. With any luck
the replica will then pick up any changes that have been made to the
master since the dump when you start slapd.
The usual disclaimer that this is just what's working for me and not a
definitive "this is how it should be done"


-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: RHEL7 issues maybe???

[Mark Cairney]

Hi Toby/Quanah,

I'm currently using a RHEL7 derivative (SL7) on our production OpenLDAP
servers i.e. Central Authorisation Service.

We use rsyslog partly out of familiarity but also because it allows
sending logs to a central rsyslog server.

The way this setup appears to work with a system using systemd + rsyslog
is that all the things are logged via systemd's journalctl.

The downside of this approach is that you are effectively running 2
logging daemons in parallel. As logging seems to be a potential
bottleneck for OpenLDAP anyway it potentially exacerbates this even further.

Another gotcha is w.r.t. rate limiting as both journald and rsyslog
implement this independently of one another. Disabling this completely
can make the bottleneck mentioned above even more apparent! Setting your
OpenLDAP logging level appropriately can mitigate this (I log at Stats+
Sync)

From my experience I'd say RHEL7 is a stable system to run OpenLDAP on.
If you have a heavily-loaded system or don't need centralised logging
though I'd try and get away with journalctl on it's own and only
introduce rsyslog logging if you need it.

Kind regards,

Mark



On 01/07/16 10:21, Toby Blake wrote:
> On Thu, 30 Jun 2016, Quanah Gibson-Mount wrote:
> 
>> On a side note, we've been moving customers off of RHEL7 back to
>> RHEL6, as we've simply found it too unstable for production use.
> 
> Hi Quanah,
> 
> This is concerning - can you provide a little more detail on this?
> 
> Cheers
> Toby Blake
> School of Informatics
> University of Edinburgh
> 

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: Odd MMR behaviour with delta-syncrepl and refreshAndPersist

[Mark Cairney]

Sure, the ITS is 8444:

http://www.openldap.org/its/index.cgi/Incoming?id=8444;page=44

I'm out the office this week anyway so won't be in a position to do any 
testing etc till next.



On 21/06/2016 02:39, Paul B. Henson wrote:

On Thu, Jun 16, 2016 at 10:10:19AM +0100, Mark Cairney wrote:


I'll fill in an ITS as suggested.

Hmm, this is on a 2.4.44 deployment with the patch from head applied
that Quanah indicated fixed the original problem he was having? I just
compiled 2.4.44 with that patch last week in preparation for an upcoming
planned upgrade; however, we use memberOf as well so now perhaps I'll
hold off again a bit . Would you be so kind as to post the ITS #
once you file it?

Thanks...




--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Odd MMR behaviour with delta-syncrepl and refreshAndPersist

[Mark Cairney]

Hi Quanah,

Apologies for taking it off-list. I didn't want to clutter the list
archive with reams of my logging output which probably isn't meaningful
to anyone else.

I'll fill in an ITS as suggested.
Thanks for your help with this.

Kind regards,

Mark


On 15/06/16 20:41, Quanah Gibson-Mount wrote:
> --On Wednesday, June 15, 2016 1:59 PM +0100 Mark Cairney
> <mark.cair...@ed.ac.uk> wrote:
> 
>> Hi Quanah,
>>
>> I can confirm I still see the issue when deleting and adding user
>> objects and groups using 3-way delta-MMR.
> 
> Please keep replies on the list.
> 
>> From one of the servers receiving the change:
>>
> 
> [snip]
> 
>> I spotted the reference to "cn=marksgroup2" in the log above so decided
>> to try it with an objectclass that has no group memberships managed by
>> the memberOf overlay (simplesecurityobject) and it worked as expected:
>>
>> Again from the consumer I see the following logged:
> 
> [snip]
> 
>> So it looks like there's possibly an additional effect being caused by
>> the memberOf overlay but as about 90% of our LDAP writes are the
>> creation/modification/deletion of users and groups this could be a pain
>> on a production system :-)
>>
>> Is this enough for you to go on? If there's any additional logging or
>> details of my config I'm happy to pass them on.
> 
> I dropped your replication log bits in case there was anything sensitive
> in there.
> 
> I agree, it looks like the memberof overlay is breaking replication in
> your case.  I would suggest filing an ITS with details on your setup,
> and the logging you provided, obfuscated as necessary.
> 
> --Quanah
> 
> 
> 
> -- 
> 
> Quanah Gibson-Mount
> Platform Architect
> Manager, Systems Team
> Zimbra, Inc.
> 
> Zimbra ::  the leader in open source messaging and collaboration
> A division of Synacor, Inc
> 

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: Odd MMR behaviour with delta-syncrepl and refreshAndPersist

[Mark Cairney]

Hi,

I see the same behaviour as before in 2.4.43- log output from one of the
servers attached. That ITS looks similar to what I'm seeing but not
entirely the same as in my case it seems to stabilise after about 5
seconds (which is still not desired behaviour).

I had a previous issue with 2.4.43 which caused one of my servers to
segfault suddenly so rolling back to that in production isn't an option
anyway.

Just in case there's anything amiss in my config my syncprov and
accesslog overlays have the following:

# {2}syncprov, {1}mdb, config
dn: olcOverlay={2}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {2}syncprov

# {3}accesslog, {1}mdb, config
dn: olcOverlay={3}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {3}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 07+00:00 01+00:00
olcAccessLogSuccess: TRUE

And the config on my accesslog DB is:

# {2}mdb, config
dn: olcDatabase={2}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {2}mdb
olcDbDirectory: /usr/local/authz/var/openldap-data/accesslog
olcSuffix: cn=accesslog
olcAccess: {0}to * by
dn.exact="uid=replicator.authorise.is.ed.ac.uk,ou=people
 ,ou=central,dc=authorise,dc=ed,dc=ac,dc=uk" write by
dn="cn=Manager,dc=author
 ise,dc=ed,dc=ac,dc=uk" write
olcLimits:
{0}dn.exact="uid=replicator.authorise.is.ed.ac.uk,ou=people,ou=cent
 ral,dc=authorise,dc=ed,dc=ac,dc=uk" time.soft=unlimited
time.hard=unlimited s
 ize.soft=unlimited size.hard=unlimited
olcLimits: {1}dn.exact="cn=Manager,dc=authorise,dc=ed,dc=ac,dc=uk"
time.soft=u
 nlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
olcRootDN: cn=Manager,cn=accesslog
olcRootPW: <-SNIP-->
olcDbIndex: default eq
olcDbIndex: entryCSN,objectClass,reqEnd,reqResult,reqStart,reqDN
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16

# {0}syncprov, {2}mdb, config
dn: olcOverlay={0}syncprov,olcDatabase={2}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpNoPresent: TRUE
olcSpReloadHint: TRUE










On 04/06/16 23:01, Paul B. Henson wrote:
> On Fri, Jun 03, 2016 at 04:06:45PM -0700, Quanah Gibson-Mount wrote:
> 
>> Likely <http://www.openldap.org/its/index.cgi/?findid=8432>
> 
> This is a new issue with 2.4.44? We've been running a 4 node MMR system
> under 2.4.43 that's been very stable and were planning to update to
> 2.4.44 this summer. Would it be better to hold off on such an update?
> 
> Thanks...
> 
> 

-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: do_syncrep2: rid=031 
cookie=rid=031,sid=004,csn=20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: slap_queue_csn: queueing 
0x4829200 20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: 
slap_graduate_commit_csn: removing 0x4829200 
20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: do_syncrep2: rid=032 
cookie=rid=032,sid=005
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: slap_queue_csn: queueing 
0x48298c0 20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: syncprov_matchops: 
skipping original sid 004
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: syncrepl_message_to_op: 
rid=031 be_add cn=marksgroup2,ou=ug,ou=iti,ou=is,dc=authorise,dc=ed,dc=ac,dc=uk 
(0)
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: slap_queue_csn: queueing 
0x48291c0 20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: syncprov_sendresp: 
to=005, cookie=rid=033,sid=006
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: 
slap_graduate_commit_csn: removing 0x48298c0 
20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: syncrepl_message_to_op: 
rid=032 be_add cn=marksgroup2,ou=ug,ou=iti,ou=is,dc=authorise,dc=ed,dc=ac,dc=uk 
(68)
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: do_syncrep2: rid=032 
delta-sync lost sync on (reqStart=20160606092722.01Z,cn=accesslog), 
switching to REFRESH
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: slap_queue_csn: queueing 
0x4829440 20160606092722.551270Z#00#004#00
Jun 6 10:27:22 rowan.authorise.is.ed.ac.uk slapd[380]: conn=1000 op=4 ABANDON 
ms

Odd MMR behaviour with delta-syncrepl and refreshAndPersist

[Mark Cairney]

e it makes sense to use
these for testing/debugging before retiring them.
-- 
/

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: Searches with dereferncing causing high CPU load.

[Mark Cairney]

On 17/11/2015 11:26, Andrew Findlay wrote:

On Tue, Nov 17, 2015 at 11:11:04AM +, Mark Cairney wrote:


Just as an update- we've managed to restore service. It turns out that
we had went over the value of 65,535 (66,291) aliases which we think was
the root cause of this behaviour suddenly starting.

It's a significant number certainly...


We're now down to "only" 41,000 :-)




Although it relates to MDB this ITS sounded very similar:
http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8146;page=10

We started deleting as many aliases as we could but performance only
improved slightly. What appears to have fixed it was doing a slapcat of
the "pruned" data and re-loading it into the database via slapadd.
Having done this searches with deref set to always are now performing as
they were before.

If this happens again, you could try stopping the server and running
slapindex rather than reloading everything.


We did try slapindex but it had little effect. This may have been before 
we'd pruned the numbers of aliases however. It's been a fraught couple 
of days...

Ultimately we've been wanting to move away from both a) hdb and b)
aliases for a while but one of our user bases runs a web application
that requires them as it doesn't support either groups or modifying it's
search filter. Given this incident there might be a push for them to
re-evaluate this approach.

That does sound like a problematic app. There may be other ways of
solving the problem if you have to keep it though. I would tend to look
at having a separate instance of slapd to service it, and it might then
be possible to use mapping overlays to build a view of your data that it
can cope with. Does the app need to modify LDAP data or is it read-only?
We had suggested that the department run their own OpenLDAP server as a 
replica of our "main" central one and do some cleverness with 
overlays/rewrites/proxies to see a subset of the objects on our server. 
We do have a number of departments who have done this, either by taking 
a feed using a script or using syncrepl + stitching together their DIT 
using overlays/subordinate databases etc.


As far as I'm aware the application itself doesn't need to write back to 
LDAP but the Administrators need write access to create their object 
structure, add new users etc.


I think the first thing I'll do is enjoy the rest of my week off then 
look at setting up a sufficiently beefy testing VM to try and reproduce 
this behaviour with a view to submitting a proper bug report.


Thanks for your help with this.

Kind regards,
Mark



Andrew



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Searches with dereferncing causing high CPU load.

[Mark Cairney]

Hi,

Just as an update- we've managed to restore service. It turns out that
we had went over the value of 65,535 (66,291) aliases which we think was
the root cause of this behaviour suddenly starting.

Although it relates to MDB this ITS sounded very similar:
http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8146;page=10

We started deleting as many aliases as we could but performance only
improved slightly. What appears to have fixed it was doing a slapcat of
the "pruned" data and re-loading it into the database via slapadd.
Having done this searches with deref set to always are now performing as
they were before.

Ultimately we've been wanting to move away from both a) hdb and b)
aliases for a while but one of our user bases runs a web application
that requires them as it doesn't support either groups or modifying it's
search filter. Given this incident there might be a push for them to
re-evaluate this approach.



On 16/11/15 18:44, Mark Cairney wrote:
> Hi Andrew,
> 
> Thanks for getting back. I saw your report for mdb actually. I can
> confirm that I've got "olcDBIndex objectlass eq" set on my servers.
> 
> Everyone keeps telling me that about aliases but unfortunately we've got
> a group of users who require them to act in lieu of groups to support
> their application i.e. they have OUs filled with aliases back to user
> accounts in the main user OU.
> 
> We've started deleting old/hanging OUs and it's made a small improvement
> but it's still taking 20-30s per query rather than returning almost
> instantly like it was before.
> 
> 
> 
> On 16/11/15 18:10, Andrew Findlay wrote:
>> On Mon, Nov 16, 2015 at 03:13:11PM +, Mark Cairney wrote:
>>
>>> We're having severe performance issues for any query with alias
>>> dereferencing set to "always".
>>>
>>> Any query with this causes the CPU to spin up to 100% and if we have a
>>> number of these concurrently the machine will become unresponsive.
>>
>> I hit something similar a while ago using mdb:
>>
>> http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8146
>>
>>> We're using OpenLDAP 2.4.42 with the old hdb backend.
>>>
>>> We do have a large number of aliases (~63,000). Could this be the cause?
>>
>> It would be worth checking that you have indexed the objectclass attribute.
>>
>> I prefer to avoid aliases...
>>
>> Andrew
>>
> 

-- 
/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Searches with dereferncing causing high CPU load.

[Mark Cairney]

Hi,

We're having severe performance issues for any query with alias
dereferencing set to "always".

Any query with this causes the CPU to spin up to 100% and if we have a
number of these concurrently the machine will become unresponsive.

We're using OpenLDAP 2.4.42 with the old hdb backend.

We do have a large number of aliases (~63,000). Could this be the cause?

Our olcMaxDerefDepth is currently set to "1"

-- 
/********

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: Searches with dereferncing causing high CPU load.

[Mark Cairney]

Hi Andrew,

Thanks for getting back. I saw your report for mdb actually. I can
confirm that I've got "olcDBIndex objectlass eq" set on my servers.

Everyone keeps telling me that about aliases but unfortunately we've got
a group of users who require them to act in lieu of groups to support
their application i.e. they have OUs filled with aliases back to user
accounts in the main user OU.

We've started deleting old/hanging OUs and it's made a small improvement
but it's still taking 20-30s per query rather than returning almost
instantly like it was before.



On 16/11/15 18:10, Andrew Findlay wrote:
> On Mon, Nov 16, 2015 at 03:13:11PM +, Mark Cairney wrote:
> 
>> We're having severe performance issues for any query with alias
>> dereferencing set to "always".
>>
>> Any query with this causes the CPU to spin up to 100% and if we have a
>> number of these concurrently the machine will become unresponsive.
> 
> I hit something similar a while ago using mdb:
> 
> http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8146
> 
>> We're using OpenLDAP 2.4.42 with the old hdb backend.
>>
>> We do have a large number of aliases (~63,000). Could this be the cause?
> 
> It would be worth checking that you have indexed the objectclass attribute.
> 
> I prefer to avoid aliases...
> 
> Andrew
> 

-- 
/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: Strange aliases behavior

[Mark Cairney]

ates: id=3585
>> first=18960 last=239706
>> Sep  4 14:53:50 ds1 slapd[4280]: mdb_search_candidates: id=3585
>> first=18960 last=239706
>> Sep  4 14:53:50 ds1 slapd[4280]: => mdb_entry_decode:
>> Sep  4 14:53:50 ds1 slapd[4280]: <= mdb_entry_decode
>> Sep  4 14:53:50 ds1 slapd[4280]: => test_filter
>> Sep  4 14:53:50 ds1 slapd[4280]: EQUALITY
>> Sep  4 14:53:50 ds1 slapd[4280]: => access_allowed: search access
>> to "cn..." "objectClass" requested
>> Sep  4 14:53:50 ds1 slapd[4280]: <= root access granted
>>
>> After that ldap start return object that is also very fast. When
>> the query was finished I saw in log this info:
>>
>>
>> Sep  4 14:53:50 ds1 slapd[4280]: mdb_search: 18985 scope not okay
>>
>> Sep  4 14:53:50 ds1 slapd[4280]: mdb_search: 18986 scope not okay
>>
>>
>> All other query that not derf. aliases are processed very fast.
>> Search time about 32k entries in subtree without aliases is about
>> 0,526s.
>>
>> Our server DB and indexing settings: 
>>
>> maxsize 10737418240
>> checkpoint 1024 10
>> sizelimit 10
>>
>> maxderefdepth 2
>> searchstack 10
>>
>> index accountid eq
>> index objectClass eq
>> index cn eq
>> index id eq
>> index name eq
>> index entryCSN eq
>> index entryUUID eq
>>
>> Do you have any idea how we can tune search with aliases? 
>> Regards 
>> Karol
> 
> My guess would be you are doing "always" for dereferencing. Try
> "search" or "find" and see if either of those meets your
> requirements and performs better. Using "always" is known to have
> this effect. 
> 
> --Quanah
> 
> 

-- 
/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



signature.asc
Description: OpenPGP digital signature

Re: Hi, I need help regarding customize schema regarding OpenLDAD 2.4.39 configuration.

[Mark Cairney]

Hi,

Coincidentally I was adding a flat .schema file to my cn=config setup 
yesterday, here’s my (rather brief) notes on the conversion process:

1. Create a temporary directory and put your .schema file in it.
mkdir /tmp/schema

2. Create a minimal slapd.conf file containing only an include of the new 
schema file

cd /tmp/schema
cp /etc/openldap/slapd.conf /tmp/slapd.conf.schema
Vim slapd.conf.schemaname

Comment everything out
Add the following line:
include /tmp/schema/schemaname.schema


Generate the schema file using slaptest while in the /tmp/schema directory :
/usr/local/authz/sbin/slaptest -f slapd.conf.schemaname  -F .

If the conversion process succeeded you should now have a cn=config/cn=schema 
directory.
In order to “ldapadd” it into an existing setup it will need a bit of 
sanitising.
Remove the commented lines and any operational/internal attributes e.g.

structuralObjectClass: olcSchemaConfig
entryUUID: dd03fc7a-d4fe-1033-96b0-055318f25a03
creatorsName: cn=config
createTimestamp: 20140920104438Z
entryCSN: 20140920104438.200261Z#00#000#00
modifiersName: cn=config
modifyTimestamp: 20140920104438Z

Modify the dn to be “cn=schemaname,cn=schema,cn=config and remove the ordering 
{0} bracket from the cn

Finally cd to the new schema location and run:
/usr/local/authz/bin/ldapadd -D cn=Manager,cn=config -W -f
cn\=\{9\}schemaname.ldif

Disclaimer: this is the process that worked for me but there are no doubt 
other/better ways to do this but slaptest is your friend here.

On 20 Sep 2014, at 07:17, Abhishek koserwal abhishek.koser...@gmail.com wrote:

 Hi, 
 
 I need some reference material regarding How to configure customize schema 
 in OpenLdap2.4.x. I have some schema files of version 2.3, when slapd.conf 
 were used. I am want to import those schema into new Openldap.2.4.39 . I have 
 gone through Admin guide tried some methods but, I didn't get much help from 
 it. Kindly help me or whom should I contact or any specific materials.
 
 Thank You,
 Abhishek koserwal,
 
 

/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk
PGP: 0x435A9621

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Hi, I need help regarding customize schema regarding OpenLDAD 2.4.39 configuration.

[Mark Cairney]

On 20 Sep 2014, at 16:48, Howard Chu h...@symas.com wrote:

 Mark Cairney wrote:
 Hi,
 
 Coincidentally I was adding a flat .schema file to my cn=config setup 
 yesterday, here’s my (rather brief) notes on the conversion process:
 
 The direct manual steps are documented in schema/openldap.ldif.

That’s pretty useful and it demonstrates that it’s probably easier to do it by 
hand than jump through the hoops I’ve described previously.

 
 1. Create a temporary directory and put your .schema file in it.
 mkdir /tmp/schema
 
 2. Create a minimal slapd.conf file containing only an include of the new 
 schema file
 
 cd /tmp/schema
 cp /etc/openldap/slapd.conf /tmp/slapd.conf.schema
 Vim slapd.conf.schemaname
 
 Comment everything out
 Add the following line:
 include /tmp/schema/schemaname.schema
 
 You'll need to include any other schemas that your to-be-converted schema 
 depends on, as well.

Good point- I’ve probably been fortunate that the only times I’ve had to add a 
schema to an existing setup the dependencies have already been there (e.g. 
edumember ) or there weren’t any.

 
 Generate the schema file using slaptest while in the /tmp/schema directory :
 /usr/local/authz/sbin/slaptest -f slapd.conf.schemaname  -F .
 
 This is the usual procedure for converting an entire configuration. If you 
 only want to convert some schema, just use:
 
 slapcat -f slapd.conf.schemaname -F /tmp/schema -n0 -s cn=schema,cn=config

Yep that’s a bit cleaner than using slaptest as it won’t output a whole 
directory structure though you’d still have to do a bit of pruning of the 
default cn=schema,cn=config stuff to get it to a state suitable for ldapadd’ing 
to a live system. 

 
 The manpages already document that any of the slap* tools can be used to 
 perform a conversion. You would know this if you read them.
 
 If the conversion process succeeded you should now have a 
 cn=config/cn=schema directory.
 
 Your conversion creates a slapd config database. As already stated countless 
 times, slapd database internal formats are subject to change without notice. 
 You should not be poking at the contents of any files within a slapd database 
 unless you really know what you're doing. If you're asking these types of 
 questions on this list, by definition you don't know what you're doing.
 
 Use the slapcat output to get the contents of a slapd database. This is why 
 the tool exists.

Well strictly speaking it creates a temporary, minimal config database purely 
for the purpose of generating the contents of the cn=schema directory. You’re 
preaching to the converted about manually hacking the config files by hand as 
even a trailing space can stop your setup from loading and having some sanity 
checking at the point of making a modification to cn=config is really useful.
However I get your point that a newbie might not appreciate the distinction 
between messing around with the contents of this temp cn=config directory and 
their own live one.

 
 On 20 Sep 2014, at 07:17, Abhishek koserwal abhishek.koser...@gmail.com 
 wrote:
 
 Hi,
 
 I need some reference material regarding How to configure customize 
 schema in OpenLdap2.4.x. I have some schema files of version 2.3, when 
 slapd.conf were used. I am want to import those schema into new 
 Openldap.2.4.39 . I have gone through Admin guide tried some methods but, I 
 didn't get much help from it. Kindly help me or whom should I contact or 
 any specific materials.
 
 Thank You,
 Abhishek koserwal,
 
 
 -- 
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/
 





-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Quick question re: memberOf and refInt overlays

[Mark Cairney]

Hi,

I suspect the answer is going to turn out to be blindingly obvious but I 
noticed this when reviewing my config of the 2 overlays mentioned and I 
thought this looked a little bit odd:


dn: olcOverlay={1}memberof,olcDatabase={1}hdb,cn=config
objectClass: olcMemberOf
objectClass: olcOverlayConfig
olcOverlay: {1}memberof
olcMemberOfDangling: ignore
olcMemberOfGroupOC: groupOfNames
olcMemberOfMemberAD: member
olcMemberOfMemberOfAD: memberOf
olcMemberOfRefInt: TRUE

dn: olcOverlay={3}refint,olcDatabase={1}hdb,cn=config
objectClass: olcConfig
objectClass: olcOverlayConfig
objectClass: olcRefintConfig
objectClass: top
olcOverlay: {3}refint
olcRefintAttribute: member
olcRefintAttribute: memberOf

So I have olcMemberOfRefint set to TRUE in the memberOf overlay and 
olcRefintAttribute: memberOf set in the refint overlay.


Is this correct or should it only be defined in one of these overlays? 
If it is defined in both overlays as it currently is, can this cause 
issues such as contention/deadlocking as there are 2 overlays 
essentially trying to do the same thing. Or does it not actually matter?

I'm using an HDB backend just in case that makes any difference.

Kind regards,

Mark

--
/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Recommended version of BDB package

[Mark Cairney]

On 07/02/14 21:01, Howard Chu wrote:



I've tested all the way up to 6.0.20. Didn't notice anything good or bad
about any of them, and all are slower than LMDB.


OK thanks for confirming this but the post on the Debian bug reporter 
list (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=688797) does state:



What version of BDB are you linked to?  There are consistent reports of
deadlock issues with BDB 5.3.  I would recommend against using that version
of BDB.  4.7.25+patches has been solid for me.  All indications with the
BDB deadlock issue in 5.3 is that it is a BDB bug, and thus nothing to do
with OpenLDAP.  It may exist in other 5.x versions of BDB.


I'm not making this up, honest ;-)


Your ITS#7657 has gotten no attention, due to lack of interest. Aliases
are a rather stupid part of LDAP; most other directory vendors don't
even bother to implement them.



If I'm honest I don't know why the people who are using them are using 
them however  my hands are tied regarding implementing a change that 
will cause a significant performance hit for them as it stands. For what 
it's worth I'd love to move to LMDB both for the performance benefits 
and the fact that it's a lot lower maintenance than HDB is!



You might be able to revive interest in that ITS by providing a complete
test case (config+data) that demonstrates the issue.



That should be do-able although as far as I can remember in the ITS I 
had demonstrated that (at least on my setup) it was reproduceable and I 
thought I'd given enough information for it to be reproduced on a clean 
test rig with a sufficiently large pool of accounts in the one OU. I'm 
also unsure of the etiquette involved in resurrecting a dormant bug.
I'm happy to give you whatever information you need though and a 
sanitised dump of my config in both HDB and MDB modes is probably a good 
place to start.


--
/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Recommended version of BDB package

[Mark Cairney]

Hi,

Having recently experienced some issues with our Live server (2.4.38, 
Scientific Linux 6.4 64-bit) I was looking into possible causes and 
found the following Debian bug report which suggests that BDB 5.X is 
problematic with OpenLDAP.


What's the best version of BDB to go with- I see 4.7.25 with patches 
mentioned in the posting and I noticed that this is the version shipped 
with LTB. If I were to downgrade my BDB database would this be the best 
one to go for?


I would like to migrate to MDB but there's an issue with alias 
de-referencing that's a bit of a deal-breaker for us at the moment as a 
number of our departments use them to provide subsets of their own 
users(ITS#7657).



Kind regards,

Mark

--
/

Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

***/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Recommended version of BDB package

[Mark Cairney]

On 07/02/2014 17:38, Quanah Gibson-Mount wrote:
--On Friday, February 07, 2014 5:17 PM + Mark Cairney 
mark.cair...@ed.ac.uk wrote:



Hi,

Having recently experienced some issues with our Live server (2.4.38,
Scientific Linux 6.4 64-bit) I was looking into possible causes and 
found

the following Debian bug report which suggests that BDB 5.X is
problematic with OpenLDAP.


Well, there are lots of 5.x releases, you'd need to be more specific 
than that.  I've used BDB 5.2.36 for years w/o issue.


Apologies, I thought I'd stated it in my first email but on reading it 
back I haven't. It's BDB 5.3.21




--Quanah


--

Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.

Zimbra ::  the leader in open source messaging and collaboration




--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Mirror mode replication breaks at times.

[Mark Cairney]

Hi,

On 08/07/2013 12:47, Pradyumna wrote:

Hi,

I have configured mirror mode replication. It's 2 node. Everything works fine 
but if I don't work on the server or say 30/40 mins or so and then when I try 
to add or delete any users or groups it don't get replicated to the other node. 
Am not getting any error in the logs and if I restart the slapd service it's 
syncs again and giving expected results.  The same setup I have in the test 
environment and its works like a charm the only difference in this setup is 
that the 2 servers are hosted on 2 different DC geographically separated where 
as in test they are in same DC.
In addition to what Quanah has said about running the latest stable 
release (there was a number of bug fixes for OpenLDAP between now and v 
2.4.23) this sounds a bit like a clock syncing/drifting issue, 
particularly if you have 2 in close proximity that work fine but the 2 
that aren't don't.


Having been bitten by this myself in the past for MMR to be reliable and 
successful the clocks on the servers have to match up almost to the 
millisecond. I'd recommend using ntpd and syncing them all to a common 
NTP time source.


I have a line like this in my /etc/ntp.conf:

server my.ntp.servers.IP minpoll 4 maxpoll 6 prefer



Am using the openldap version which comes by default with RHEL 6.3. If it would 
have been a version issue then I should have expected the same result in test 
as well? Please help.


Kind regards,

Mark


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Problems getting mdb_stat to work

[Mark Cairney]

Hi,

I've acquired the libmdb tools from the Gitorious page. I've got it to compile 
(with some warnings, see below) but it throws an error when I try and run 
mdb_stat on the mdb directory:

Compilation yields the following:

[root@birch libmdb]# ./make.sh 
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic -fPIC  -c mdb.c
mdb.c:433: warning: extra semicolon in struct or union specified
mdb.c:796: warning: implicit declaration of function ‘fdatasync’
mdb.c:1388: warning: implicit declaration of function ‘pwrite’
mdb.c:1699: warning: implicit declaration of function ‘ftruncate’
mdb.c:1885: warning: implicit declaration of function ‘strdup’
mdb.c:1885: warning: assignment makes pointer from integer without a cast
mdb.c:4079: warning: assignment makes pointer from integer without a cast
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic -fPIC  -c midl.c
ar rs libmdb.a mdb.o midl.o
ar: creating libmdb.a
gcc -shared -o libmdb.so mdb.o midl.o
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  -c mdb_stat.c
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  mdb_stat.o libmdb.a  -o mdb_stat
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  -c mtest.c
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  mtest.o libmdb.a  -o mtest
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  -c mtest2.c
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  mtest2.o libmdb.a  -o mtest2
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  -c mtest3.c
gcc -pthread -O2 -g -DMDB_DSYNC=O_SYNC -W -Wall -Wno-unused-parameter 
-Wbad-function-cast -std=c99 -pedantic  mtest3.o libmdb.a  -o mtest3

Running the command gives the following:

[root@birch tmp]# ./mdb_stat /usr/local/authz/var/openldap-data/authorise-test
mdb_env_open failed, error 22

Running this through strace looks like:

[root@birch tmp]# strace ./mdb_stat 
/usr/local/authz/var/openldap-data/authorise-test
execve(./mdb_stat, [./mdb_stat, /usr/local/authz/var/openldap-da...], [/* 
29 vars */]) = 0
brk(0)  = 0x200f000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7fb138ef5000
access(/etc/ld.so.preload, R_OK)  = -1 ENOENT (No such file or directory)
open(/etc/ld.so.cache, O_RDONLY)  = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=66956, ...}) = 0
mmap(NULL, 66956, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb138ee4000
close(3)= 0
open(/lib64/libpthread.so.0, O_RDONLY) = 3
read(3, \177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\0\1\0\0\0`\\\340\2401\0\0\0..., 
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=145720, ...}) = 0
mmap(0x31a0e0, 2212768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 
0) = 0x31a0e0
mprotect(0x31a0e17000, 2097152, PROT_NONE) = 0
mmap(0x31a1017000, 8192, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x31a1017000
mmap(0x31a1019000, 13216, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31a1019000
close(3)= 0
open(/lib64/libc.so.6, O_RDONLY)  = 3
read(3, 
\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\0\1\0\0\0\360\355a\2401\0\0\0..., 832) = 
832
fstat(3, {st_mode=S_IFREG|0755, st_size=1922112, ...}) = 0
mmap(0x31a060, 3745960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 
0) = 0x31a060
mprotect(0x31a0789000, 2097152, PROT_NONE) = 0
mmap(0x31a0989000, 20480, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x189000) = 0x31a0989000
mmap(0x31a098e000, 18600, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31a098e000
close(3)= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7fb138ee3000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7fb138ee2000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7fb138ee1000
arch_prctl(ARCH_SET_FS, 0x7fb138ee2700) = 0
mprotect(0x31a0989000, 16384, PROT_READ) = 0
mprotect(0x31a1017000, 4096, PROT_READ) = 0
mprotect(0x31a041f000, 4096, PROT_READ) = 0
munmap(0x7fb138ee4000, 66956)   = 0
set_tid_address(0x7fb138ee29d0) = 13054
set_robust_list(0x7fb138ee29e0, 0x18)   = 0
futex(0x7fff78256ffc, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7fff78256ffc, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 
7fb138ee2700) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x31a0e05ae0, [], SA_RESTORER|SA_SIGINFO, 
0x31a0e0f500}, NULL, 8) = 0

Re: Problems getting mdb_stat to work

[Mark Cairney]

On 27/11/2012 19:29, Howard Chu wrote:


Your first mistake was not actually reading the gitorious page. Go 
read it again.


The OpenLDAP source tree already includes the MDB source code.



D'oh! Indeed sitting in my openldap source directory there is a 
libraries/libmdb directory with a Makefile in it.


Can't believe I missed that the first time round. Anyway that's mdb_stat 
now up and running.


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: ldapsearch stalling when performing searches with a large number of results

[Mark Cairney]

On 26 Sep 2012, at 22:36, Quanah Gibson-Mount wrote:

 --On Wednesday, September 26, 2012 11:59 AM +0100 Mark Cairney 
 mark.cair...@ed.ac.uk wrote:
 
 My olcDB values are listed below (minus the olcDbConfig entries). I'm not
 sure if you need the indexes but I've left them in anyway:
 
 
 olcDbCacheFree: 1
 olcDbCacheSize: 40
 olcDbIDLcacheSize: 120
 
 set_cachesize 4 0 1
 
 This may be a little small. I prefer to fully cache my DB.

Which one in particular? I thought the set_cachesize had an upper limit of 4GB 
but your guidance on the Zimbra website suggests this is an old limit and I now 
can't find the page on the OpenLDAP site which discusses it. Alternatively 
would increasing the number of caches from 2 to 2 or 3 be a suitable workaround?

Given that I've got a relatively healthy amount of RAM available would the 
following sound sensible to you

olcDbCacheSize: 100
olcDbIDLcacheSize: 120
set_cachesize 4 0 2

I also want to give a bit of headroom for new user accounts (approx rate of 
increase 80,000/y) and creating a group object for each user.


 
 
 I'm not using an SHM key (should I be?).
 
 So with 300,000 users, your caches look fine.  I would definitely recommend 
 using an SHM key if you are going to stick with using BDB.  I personally 
 prefer using MDB with current RE24 these days. It is magnitudes faster than 
 BDB in all aspects if you enable write map.

I've looked at your guidance on using SHM keys but I'm slightly reluctant to 
start playing around with kernel settings on production servers :-) 
The existing default settings on RHEL 5 seem massive in comparison though:
kernel.shmmax = 68719476736

kernel.shmall = 4294967296

Whereas based on the ZImbra performance tuning page I calculated (based on 8GB 
BDB cache size + 0.5GB for other stuff)

shmall would be: 2228224

and shmmax:  8589934592

Both of which appear to be an order of magnitude smaller than the defaults!
Then there appears to be some Zimbra-specific commands but I'm guessing the 
equivalent is just setting the olcDBShmKey in slapd.d on vanilla OpenLDAP?

My plan in the longer term is to move to MDB but when I tried it out on one of 
our test VMs (40GB HD) it pretty much devoured all available disk space. Is 
there a rule of thumb for deriving probable MDB disk space requirements based 
on existing BDB size?

Thanks for the help and apologies for all the additional questions!

Kind regards,

Mark



-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: ldapsearch stalling when performing searches with a large number of results

[Mark Cairney]

My olcDB values are listed below (minus the olcDbConfig entries). I'm not sure 
if you need the indexes but I've left them in anyway:

olcDbDirectory: /usr/local/authz/var/openldap-data/authorise
olcAddContentAcl: FALSE
olcDbCacheFree: 1
olcDbCacheSize: 40
olcDbCheckpoint: 10240 10

olcDbDirtyRead: FALSE
olcDbDNcacheSize: 0
olcDbIDLcacheSize: 120
olcDbIndex: autoMountKey eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniOrgCode eq
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: eduniType eq
olcDbIndex: eduniUnitCode eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: eduPersonPrimaryAffiliation eq
olcDbIndex: eduPersonPrincipalName eq
olcDbIndex: eduPersonScopedAffiliation pres,eq
olcDbIndex: eduPersonTargetedID eq
olcDbIndex: entryCSN eq
olcDbIndex: entryUUID eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: gidNumber pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: memberOf pres,eq
olcDbIndex: memberUid eq
olcDbIndex: objectClass eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: uniqueMember pres,eq
olcDbIndex: userPassword eq
olcDbLinearIndex: FALSE
olcDbMode: 0600
olcDbNoSync: FALSE
olcDbSearchStack: 16
olcDbShmKey: 0

and my DB_CONFIG file contains:

set_cachesize 4 0 1
set_flags DB_LOG_AUTOREMOVE
set_lk_max_objects 5000
set_lk_max_lockers 5000
set_lg_regionmax 41943040
set_lg_dir /usr/local/authz/slapd/trans-logs
set_lk_max_locks 1
set_lg_bsize 20971520
set_lg_max 83886080

I'm not using an SHM key (should I be?).

The search that I spotted this behaviour probably means nothing outside our 
environment as it's against a value in our local schema but for completeness it 
was:

((eduniCategory=201)(objectclass=inetorgperson))

I've seen the same behaviour with similar searches which we would expect to 
return a large number of results e.g. 
((eduPersonAffiliation=student)(objectclass=inetorgperson)) or indeed
against groups with a lot of members.

 
 You may wish to read over 
 https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning.  You can easily 
 map the information there to a non-zimbra installation.

Thanks- your documentation on the DB tuning and SHM key is excellent. I can see 
a couple of things which I might change based on it but I'll wait to hear back 
before I jump in and start playing around with parameters.

 
 --Quanah
 
 --
 
 Quanah Gibson-Mount
 Sr. Member of Technical Staff
 Zimbra, Inc
 A Division of VMware, Inc.
 
 Zimbra ::  the leader in open source messaging and collaboration
 

/

Mark R Cairney
ITI UNIX Section
Information Services

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

ldapsearch stalling when performing searches with a large number of results

[Mark Cairney]

Hi,

I've been finding that ldapsearch will freeze when performing searches with a 
large (typically 10,000) number of results.
As far as I can tell this does not appear to be related to server load as 
according to iostat and top the servers don't appear to be stressed
and the servers continue to respond to incoming requests.

I'm running OpenLDAP 2.4.30 with BDB 4.8.30 in multi-master mode with syncrepl 
if this makes any difference. I have tried upgrading to 2.4.32 but 
this appears to be exhibiting the same behaviour.

In terms of the size of the database I've got about 300,000 entries and my bdb 
database files are currently taking up 5.6G.
The servers themselves are 4xquad-cores with 24GB RAM.


Has anyone else seen this behaviour or should I file an ITS?
/

Mark R Cairney
ITI UNIX Section
Information Services

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Problem with olcAccess

[Mark Cairney]

Hi Tobias,

It looks like it's simply base64 encoded. Did you have any trailing 
whitespace after your olcAccess entry? As far as I'm aware the rule will 
still be processed correctly although I'm happy to be corrected if this 
isn't the case.


Cheers,

Mark

On 22/09/12 13:47, Tobias Hachmer wrote:

Hello list,

I simply trying to add an olcAccess entry to the config backend.

here the file contents:

dn: olcDatabase={1}hdb,cn=config
changeType: modify
add: olcAccess
olcAccess: to dn.subtree=ou=public,ou=addressbook,dc=example,dc=com by
users write

What I've get after adding this to the backend is:

olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by
dn=cn=a
  dmin,dc=example,dc=com write by * none
olcAccess: {1}to dn.base= by * read
olcAccess: {2}to * by self write by dn=cn=admin,dc=example,dc=com
write by *
   read
olcAccess::
ezN9dG8gZG4uc3VidHJlZT0ib3U9cHVibGljLG91PWFkZHJlc3Nib29rLGRjPWtva2
  VsbmV0LGRjPWRlIiBieSAqIHdyaXRlIA==

What's going on here, what did I wrong, I didn't get it yet. Please help
me.

Regards,
Tobias Hachmer





--
/

Mark Cairney
ITI UNIX Section
Information Services

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

/

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Syncrepl issues, glue objectClass

[Mark Cairney]

Hi,

Just for completeness the issue turned out to be time synchronisation- one of 
the 3 nodes was still going to RHN for it's NTP sync rather than our local NTP 
server which the other 2 were pointed at. I'm guessing that there wasn't enough 
of a difference for the sync cookie to be rejected but enough of a time skew 
that they were received in the wrong order although this is pure conjecture on 
my part and am happy to be proven wrong.
 They now appear to be a lot happier!

Thanks,

Mark

On 10 Jun 2011, at 16:30, Quanah Gibson-Mount wrote:

 --On Friday, June 10, 2011 3:04 PM +0100 Mark Cairney mark.cair...@ed.ac.uk 
 wrote:
 
 Hi,
 
 I've been trying to use a tool called Grouper to provision a
 hierarchical structure into my LDAP directory.
 
 I'm currently running OpenLDAP 2.4.25 with BDB 4.8.30 on 3 SL5.5 servers
 in a multi-master configuration.
 
 During the provisioning process it seems to be hitting a race condition
 where it creates a higher-level ou before the base-level ou is there
 resulting in the base-level ou existing in the tree with the glue
 objectClass.
 
 As this is invisible to searches I end up with syncrepl constantly trying
 to replicate it ad infinitum:
 
 I would suggest filing an ITS with configurations and exact instructions on 
 how to reproduce the issue at http://www.openldap.org/its/
 
 --Quanah
 
 
 --
 
 Quanah Gibson-Mount
 Sr. Member of Technical Staff
 Zimbra, Inc
 A Division of VMware, Inc.
 
 Zimbra ::  the leader in open source messaging and collaboration
 

/* 
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Syncrepl issues, glue objectClass

[Mark Cairney]

Hi,

I've been trying to use a tool called Grouper to provision a hierarchical 
structure into my LDAP directory.

I'm currently running OpenLDAP 2.4.25 with BDB 4.8.30 on 3 SL5.5 servers in a 
multi-master configuration.

During the provisioning process it seems to be hitting a race condition where 
it creates a higher-level ou before the base-level ou is there resulting in the 
base-level ou existing in the tree with the glue objectClass.

As this is invisible to searches I end up with syncrepl constantly trying to 
replicate it ad infinitum:

Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=031 be_modify 
ou=2010/2011,ou=courses,ou=grouper,dc=authorise,dc=ed,dc=ac,dc=uk (0) 
Jun 10 14:53:16 alder slapd[20702]: syncprov_sendresp: cookie= 
Jun 10 14:53:16 alder slapd[20702]: do_syncrep2: rid=032 cookie= 
Jun 10 14:53:16 alder slapd[20702]: do_syncrep2: rid=031 cookie= 
Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=032 
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) 
Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=032 be_search (0) 
Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=032 
ou=2010/2011,ou=courses,ou=grouper,dc=authorise,dc=ed,dc=ac,dc=uk 
Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=031 
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) 
Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=031 be_search (0) 
Jun 10 14:53:16 alder slapd[20702]: syncrepl_entry: rid=031 
ou=2010/2011,ou=courses,ou=grouper,dc=authorise,dc=ed,dc=ac,dc=uk 
Jun 10 14:53:16 alder slapd[20702]: syncprov_sendresp: cookie= 
Jun 10 14:53:17 alder last message repeated 210 times
Jun 10 14:53:17 alder slapd[20702]: syncrepl_entry: rid=032 be_modify 
ou=2010/2011,ou=courses,ou=grouper,dc=authorise,dc=ed,dc=ac,dc=uk (0) 
Jun 10 14:53:17 alder slapd[20702]: do_syncrep2: rid=032 cookie= 
Jun 10 14:53:17 alder slapd[20702]: syncrepl_entry: rid=032 
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) 

Doing a bit of digging on the web the suggested solution is to modify the 
object and remove the glue objectlcass but there doesn't seem to be an obvious 
way to do this given that it's invisible to ldapsearch and if you try an 
ldapadd the object already exists?

Even a sample ldif file for ldapmodify would be grand.

Cheers,

Mark


/* 
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Syncrepl issues, glue objectClass

[Mark Cairney]

Hi Quanah,

Thanks for the reply- in the meantime is there any way I can apply some 
first-aid to work around the issue and make the ghost OUs appear as expected?
I've also asked a similar question on the Grouper mailing list as it may be a 
case of garbage in, garbage out.

The behaviour of OpenLDAP is a little bit odd as well though so I'll file an 
ITS.

Cheers,

Mark

On 10 Jun 2011, at 16:30, Quanah Gibson-Mount wrote:

 --On Friday, June 10, 2011 3:04 PM +0100 Mark Cairney mark.cair...@ed.ac.uk 
 wrote:
 
 Hi,
 
 I've been trying to use a tool called Grouper to provision a
 hierarchical structure into my LDAP directory.
 
 I'm currently running OpenLDAP 2.4.25 with BDB 4.8.30 on 3 SL5.5 servers
 in a multi-master configuration.
 
 During the provisioning process it seems to be hitting a race condition
 where it creates a higher-level ou before the base-level ou is there
 resulting in the base-level ou existing in the tree with the glue
 objectClass.
 
 As this is invisible to searches I end up with syncrepl constantly trying
 to replicate it ad infinitum:
 
 I would suggest filing an ITS with configurations and exact instructions on 
 how to reproduce the issue at http://www.openldap.org/its/
 
 --Quanah
 
 
 --
 
 Quanah Gibson-Mount
 Sr. Member of Technical Staff
 Zimbra, Inc
 A Division of VMware, Inc.
 
 Zimbra ::  the leader in open source messaging and collaboration
 

/* 
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

MemberOf attribute not being returned

[Mark Cairney]

Hi,

I'm sure this was working in the past on this server but Im now not getting 
anything returned when I request the memberOf attribute.

I compiled OpenLDAP 2.4.23 with the following flags:

./configure --prefix=/usr/local/authz --enable-meta --enable-ldap --enable-bdb 
--enable-monitor --enable-syncprov --enable-translucent --enable-memberof 
--enable-dyngroup --enable-dynlist --with-threads --with-tls --with-cyrus-sasl 
--enable-syslog --enable-spasswd cd  make depend make make test make install

I'm using slapd.d and I have the following in 
/usr/local/authz/etc/openldap/slapd.d/cn=config/olcDatabase={1}bdb

olcOverlay={0}dynlist.ldif
olcOverlay={1}memberof.ldif
olcOverlay={2}syncprov.ldif

The contents of olcOverlay\=\{1\}memberof.ldif are:

dn: olcOverlay={1}memberof
objectClass: olcOverlayConfig
objectClass: olcMemberOf
olcMemberOfDangling: ignore
olcMemberOfRefInt: FALSE
olcMemberOfGroupOC: posixGroup
olcMemberOfMemberAD: member
olcMemberOfMemberOfAD: memberOf
structuralObjectClass: olcMemberOf
entryUUID: 4d5a3aa8-fbac-45c9-b259-941d13e02724
creatorsName: cn=config
createTimestamp: 20100318151149Z
entryCSN: 20100318151149.488341Z#00#003#00
modifiersName: cn=config
modifyTimestamp: 20100318151149Z
olcOverlay: {1}memberof


The log is attached.

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



openldap.log
Description: Binary data


Any ideas? The only thing I've changed recently is the ACLs

Kind regards,

Mark

/* 
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/

RE: MemberOf attribute not being returned

[Mark Cairney]

Nevermind, I think I know what's happening. My user account was updated  on our 
current live server running OpenLDAP 2.3 which doesn't have the MemberOf 
overlay.

When this change was applied using syncrepl the memberOf field must have been 
removed.

I'll take the old server out of the syncrepl  but in the meantime is there any 
way to ensure this field is preserved when provisioning accounts in LDAP?

Kind regards,

Mark

/* 
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: MemberOf attribute not being returned

[Mark Cairney]

Warning duly noted. Lessing the files in question seemed like the best way of 
providing a concise description of what configuration I had and where in the 
config it lay.
As it stands I answered my question anyway.

Kind regards,

Mark

On 27 Jan 2011, at 12:16, Howard Chu wrote:

 Mark Cairney wrote:
 Hi,
 
 I'm sure this was working in the past on this server but Im now not getting 
 anything returned when I request the memberOf attribute.
 
 I compiled OpenLDAP 2.4.23 with the following flags:
 
 ./configure --prefix=/usr/local/authz --enable-meta --enable-ldap 
 --enable-bdb --enable-monitor --enable-syncprov --enable-translucent 
 --enable-memberof --enable-dyngroup --enable-dynlist --with-threads 
 --with-tls --with-cyrus-sasl --enable-syslog --enable-spasswd cd  make 
 depend make make test make install
 
 I'm using slapd.d and I have the following in 
 /usr/local/authz/etc/openldap/slapd.d/cn=config/olcDatabase={1}bdb
 
 olcOverlay={0}dynlist.ldif
 olcOverlay={1}memberof.ldif
 olcOverlay={2}syncprov.ldif
 
 The contents of olcOverlay\=\{1\}memberof.ldif are:
 
 You should not be poking or peeking at the files inside slapd.d. You should 
 be using slapcat -n0 or ldapsearch -b cn=config to show the contents of 
 the config database. As with other slapd databases, its structure and format 
 are subject to change without notice at any time. The only thing guaranteed 
 to remain compatible is the LDAP interfaces to the database.
 
 -- 
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/
 

/* 
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Blank node in multi-master OpenLDAP 2.4.21 setup (findbase failed)

[Mark Cairney]

Hi,

Apache Directory Studio appears to have FUBAR'ed one of the nodes in our 
multi-master OpenLDAP setup and I'd appreciate some help or pointers.

Im running OpenLDAP 2.4.21 and BDB 4.8.26 with Kerberos 5 and GSSAPI SASL.

On the afflicted node the DIT is empty other than the Root DSE and ldapsearch 
returns 32 no such object.

The logs contain the following:

** ld 0x2aab4c4b7c70 Outstanding Requests:
connection_get(44): got connid=1095
connection_read(44): checking for input on id=1095
 * msgid 4,  origid 4, status InProgress
   outstanding referrals 0, parent count 0
ldap_pvt_sasl_generic_install
  ld 0x2aab4c4b7c70 request count 1 (abandoned 0)
** ld 0x2aab4c4b7c70 Response Queue:
ber_get_next
   Empty
  ld 0x2aab4c4b7c70 response count 0
ldap_chkResponseList ld 0x2aab4c4b7c70 msgid 4 all 0
ldap_chkResponseList returns ld 0x2aab4c4b7c70 NULL
ldap_int_select
ber_get_next: tag 0x30 len 292 contents:
op tag 0x63, time 1286205781
ber_get_next
conn=1095 op=3 do_search
ber_scanf fmt ({mb) ber:
 dnPrettyNormal: dc=authorise,dc=ed,dc=ac,dc=uk
 dnPrettyNormal: dc=authorise,dc=ed,dc=ac,dc=uk, 
dc=authorise,dc=ed,dc=ac,dc=uk
ber_scanf fmt (m) ber:
ber_scanf fmt ({M}}) ber:
= get_ctrls
ber_scanf fmt ({m) ber:
ber_scanf fmt (m) ber:
= get_ctrls: oid=1.3.6.1.4.1.4203.1.9.1.1 (noncritical)
ber_scanf fmt ({i) ber:
ber_scanf fmt (m) ber:
ber_scanf fmt (b) ber:
ber_scanf fmt (}) ber:
ber_scanf fmt ({m) ber:
ber_scanf fmt (b) ber:
= get_ctrls: oid=2.16.840.1.113730.3.4.2 (critical)
= get_ctrls: n=2 rc=0 err=
== limits_get: conn=1095 op=3 
self=uid=replicator.authorise.is.ed.ac.uk,ou=people,ou=central,dc=authorise,dc=ed,dc=ac,dc=uk
 this=dc=authorise,dc=ed,dc=ac,dc=uk
= bdb_search
bdb_dn2entry(dc=authorise,dc=ed,dc=ac,dc=uk)
bdb_dn2entry(cn=admins,ou=group,ou=central,dc=authorise,dc=ed,dc=ac,dc=uk)
bdb_entry_get: rc=0
send_ldap_result: conn=1095 op=3 p=3
findbase failed! 32
send_ldap_result: conn=1095 op=3 p=3
send_ldap_response: msgid=4 tag=101 err=32
ber_flush2: 14 bytes to sd 44
connection_get(43): got connid=0
=do_syncrepl rid=030
=do_syncrep2 rid=030
ldap_result ld 0x2aab4c4b7c70 msgid 4
wait4msg ld 0x2aab4c4b7c70 msgid 4 (timeout 0 usec)
wait4msg continue ld 0x2aab4c4b7c70 msgid 4 all 0
** ld 0x2aab4c4b7c70 Connections:
* host: alder.authorise.is.ed.ac.uk  port: 636  (default)
  refcnt: 2  status: Connected
  last used: Mon Oct  4 16:23:01 2010


** ld 0x2aab4c4b7c70 Outstanding Requests:
 * msgid 4,  origid 4, status InProgress
   outstanding referrals 0, parent count 0
  ld 0x2aab4c4b7c70 request count 1 (abandoned 0)
** ld 0x2aab4c4b7c70 Response Queue:
   Empty
  ld 0x2aab4c4b7c70 response count 0
ldap_chkResponseList ld 0x2aab4c4b7c70 msgid 4 all 0
ldap_chkResponseList returns ld 0x2aab4c4b7c70 NULL
ldap_int_select
read1msg: ld 0x2aab4c4b7c70 msgid 4 all 0
ber_get_next
ber_get_next: tag 0x30 len 12 contents:
read1msg: ld 0x2aab4c4b7c70 msgid 4 message type search-result
ber_scanf fmt ({eAA) ber:
read1msg: ld 0x2aab4c4b7c70 0 new referrals
read1msg:  mark request completed, ld 0x2aab4c4b7c70 msgid 4
request done: ld 0x2aab4c4b7c70 msgid 4
res_errno: 32, res_error: , res_matched: 
ldap_free_request (origid 4, msgid 4)
ldap_parse_result
ber_scanf fmt ({iAA) ber:
ber_scanf fmt (}) ber:
ldap_err2string
do_syncrep2: rid=030 LDAP_RES_SEARCH_RESULT (32) No such object
ldap_err2string
ldap_err2string
do_syncrep2: rid=030 (32) No such object
ldap_err2string
ldap_msgfree
connection_get(43): got connid=0
ldap_free_connection 1 1
ldap_send_unbind
ber_flush2: 7 bytes to sd 43
connection_get(44): got connid=1095
connection_read(44): checking for input on id=1095
ber_get_next
TLS trace: SSL3 alert write:warning:close notify
ldap_free_connection: actually freed
ber_get_next: tag 0x30 len 5 contents:
op tag 0x42, time 1286205781
ber_get_next
do_syncrepl: rid=030 rc -2 retrying
TLS trace: SSL3 alert read:warning:close notify
ber_get_next on fd 44 failed errno=0 (Success)
conn=1095 op=4 do_unbind
connection_close: conn=1095 sd=44
TLS trace: SSL3 alert write:warning:close notify

So far to fix it I've tried running slapd with the -c rid= option, deleting 
the contents of /var/openldap-data, running db_verify and db_recover (with and 
without the -c flag) and doing a slapadd from one of the other working nodes 
but nothing has worked.

Interestingly the file sizes in /var/openldap-data/authorise look OK but the 
LDAP tree appears to have vanished without a trace.

Any ideas?

Kind regards,

Mark


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.