2011/3/25 Cannady, Mike <[email protected]>
>
>
> > -----Original Message-----
> > From: Buchan Milne [mailto:[email protected]]
> > Sent: Friday, March 25, 2011 6:17 AM
> > To: Cannady, Mike
> > Cc: [email protected]
> > Subject: Re: Multimaster ldap related questions
> >
> >
> > ----- "Mike Cannady" <[email protected]> wrote:
> >
> > > I have implemented a multi-master two node ldap with openldap 2.4.22
> > > and Berkely DB 4.8.26 on Redhat enterprise 5.4 with several readonly
> > > replicas off of the masters.
> > >
> > > I have a need to add several optional attributes to a schema and
> > > probably should upgrade to 2.4.24 as well. If this was a
> > > single-master
> > > server, it would be easy to do; just slapcat the ldap store, update
> > > software, change schema, slapadd the ldap store back, and resume
> > > slapd.
> >
> > Why would you need to slapcat/slapadd to "add several optional
> attributes"
>
> While testing the 2 node multi-master nodes, I did identical changes
> (added an optional attribute to a schema) by stopping the 2 slapd
> daemonds, changed schema, started the daemonds. After a lot of adding
> and deleting of info out of the stores, it was apparent that something
> was wrong with the data and the syncs. I stopped both daemons, did the
> dump and restore on one and purge the database on the other and started
> them back up. I didn't have any problems after that. I know I didn't
> have issues prior to the changes, so I assumed the S.O.P. for schema
> changes is dump, change, restore.
>
> Are you indicating that isn't the case? What about newly defined
> required attributes in a schema or one that was optional and now
> required where all the instances have the optional already specified.
>
> >
> > > I'm not sure how to do that with multi-master. One reason for using
> > > multi-master was if one master was down, the other would keep
> > > running.
> > > One should be able to upgrade one server, have it catch up with the
> > > changes that the other master had done while the first master is
> down
> > > and then repeat for the 2nd master.
> >
> > Well, it would apply if you weren't modifying data offline on the 1st
> > master.
> >
> > > Is this possible? Has anyone
> > > done
> > > this and how was it done?
> > >
> > > I know in the near future, a high-level branch on my DIT will be
> > > purged
> > > and bulk reloaded.
> >
> > I can't think of a strategy where a bulk load will have neither:
> > -write downtime
> > -inconsistency (changes made in the window between the bulk generation
> and
> > the startup of the server after import will be lost)
> >
> > You aren't clear which of these you want/prefer/require.
> >
>
> I want to minimize the time that the data in the store is unavailable.
> If the delete and bulk-load take 10 hours to do online, then the service
> is effectively down for 10 hours. If the offline version of dump,
> delete what is to be deleted, slapadd the results, and then slapadd the
> new info takes only 30-60 minutes. The offline would be the method I
> choose based on time. The amount of time was my next question:
>
>
> > > I have tested the load with a test setup of
> > > multi-master ldap. If I do it via ldapadd, it takes over 6 hours to
> > > load. With slapadd (and slapd down) it only takes 25 minutes plus
> > > the
> > > time for the other master to get up-to-date.
> >
> > What is tool-threads set to? Which interesting slapadd options (e.g.
> -q)
> > did you use?
>
> Tool-threads is not specified, so I guess its one. The test box is a
> single hyperthreaded cpu.
>
> Slapadd command (for master # 1)
> slapadd -c -f ldapfile -n 1 -S 001 -w
>
>
> >
> > > Is there any way that I
> > > can speed-up the update with ldapadd?
> >
> > ldapadd will never be as fast as slapadd.
> >
>
> Granted. What I'm asking is there a way to speed up slapd for the
> duration of the mass deletes and bulk loads. In normal circumstances,
> the syncs and such would be OK for normal processing; but, with bulk
> changes, I would like have the daemon run in such a way that syncs are
> not done and everything is done in memory. My tests seem to indicate
> that disk I/O is the bottleneck. I know if there is a failure during
> this time, the database may be corrupted, but this OK for the bulk
> change duration.
>
Hello,
I have a machine that was time consuming to restore a base
ldif with "slapadd.
My problem was the I / O (HD very slow).
Resolved with a small shell:
#!/bin/bash
/etc/init.d/slapd stop
slapcat /tmp/backup.ldif
dir_ldap="/var/lib/ldap"
rm -f /var/lib/ldap/{alock,__db.*,*bdb,log.*}
mv $dir_ldap $dir_ldap.real
mkdir $dir_ldap
mount -t tmpfs tmpfs $dir_ldap -o nr_inodes=3000000,size=4G
cp $dir_ldap.real/DB_CONFIG $dir_ldap
slapadd -c -q -w -b "dc=example,dc=com" -l /tmp/backup.ldif
rsync -a --delete $dir_ldap/ $dir_ldap.real/
umount $dir_ldap
rmdir $dir_ldap
mv $dir_ldap.real $dir_ldap
chown openldap:openldap $dir_ldap -R
/etc/init.d/slapd start
Thus, the I / O during the slapadd no longer exists, because all files are
in memory. The time difference with "tmpfs" is striking.
Regards,
Jarbas
>
>
> > > I have pieces of my slapd.conf
> > > for the 1st master at the end of this email.
> > >
> > > Slapadd has two options that appear to be needed when dealing with
> > > multi-master or replicate nodes. The first is the "-S sid" option,
> > > the
> > > second is "-w". I'm a little confused what is used where. If you
> > > are
> > > doing a dump and restore operation (slapcat, delete database,
> > > slapadd)
> > > the only option you need is the "-w" option? If you are adding new
> > > entries offline then do both options need to be specified?
> >
> > Adding, *or* modifying.
>
> I don't understand this answer.
>
> >
> > > Is there a multi-master best practice quide somewhere?
> >
> > A good start is to never lie to slapd. If you have changed the
> contents of
> > an entry, the entryCSN should not be retained.
> >
> > I also prefer to avoid non-restore bulk-loading.
>
> Me too, but if it is a decision of 10 hours vs. 1 hour, the 1 hour will
> win.
>
>
> >
> > Regards,
> > Buchan
>
>
> Thanks,
>
> Mike
>
> **********************************************************************
> HTC Disclaimer: The information contained in this message may be
> privileged and confidential and protected from disclosure. If the reader of
> this message is not the intended recipient, or an employee or agent
> responsible for delivering this message to the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, please notify us immediately by replying to the
> message and deleting it from your computer. Thank you.
> **********************************************************************
>
>
>
