Re: Using virtual IP and N-way mutlimaster mode

[Quanah Gibson-Mount]

--On Wednesday, January 24, 2018 6:40 PM + Andrew Findlay 
 wrote:



I'll discuss with Howard, and see.  I hate seeing more options added to
slapd, but it may be the only option (no pun intended!) for this
scenario. ;)  The cloud was fairly nascent when this was all designed,
so this case wasn't really a consideration at that point in time.  If
you were to pass in the server ID, I think you could get rid of the
multi-valued serverID bit entirely, since every server would know who it
was already.


I think you would still need that. Maybe I have missed something else
here, but how does slapd avoid making a syncrepl connection to itself
in the replicated-config scenario? I was assuming that it just ignores
syncrepl clauses where the provider URL matches its own hostname.
If it is done by IP then more thought will be required.


I'm not saying to get rid of it entirely, I'm just noting that with it 
passed in as an argument, there shouldn't be a need to have a multivalued 
setting in the configuration.  In fact, no serverID setting in the config 
at all.  Even if it connected to itself, it would discard all traffic that 
came from itself (by checking the serverID value in the entryCSN, which is 
how loops are currently avoided when a change originating on a master makes 
its way back from another master).  Might add some overhead though.


--Quanah


--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

Re: Using virtual IP and N-way mutlimaster mode

[Andrew Findlay]

On Wed, Jan 24, 2018 at 09:43:58AM -0800, Quanah Gibson-Mount wrote:

> > How about adding a flag to slapd to specify the server ID so that it can
> > go the other way through the config, convert ID to FQDN, and drop that
> > FQDN from the set of replication sources?
> 
> I'll discuss with Howard, and see.  I hate seeing more options added to
> slapd, but it may be the only option (no pun intended!) for this scenario.
> ;)  The cloud was fairly nascent when this was all designed, so this case
> wasn't really a consideration at that point in time.  If you were to pass in
> the server ID, I think you could get rid of the multi-valued serverID bit
> entirely, since every server would know who it was already.

I think you would still need that. Maybe I have missed something else
here, but how does slapd avoid making a syncrepl connection to itself
in the replicated-config scenario? I was assuming that it just ignores
syncrepl clauses where the provider URL matches its own hostname.
If it is done by IP then more thought will be required.

The multi-valued attribute is neat in the case of servers with simple
IP<->hostname mappings so we would probably want to keep it for that.

Andrew
-- 
---
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/+44 1628 782565 |
---

Re: Using virtual IP and N-way mutlimaster mode

[Michael Ströder]

Andrew Findlay wrote:
> On Wed, Jan 24, 2018 at 09:43:58AM -0800, Quanah Gibson-Mount wrote:
> 
>>> How about adding a flag to slapd to specify the server ID so that it can
>>> go the other way through the config, convert ID to FQDN, and drop that
>>> FQDN from the set of replication sources?
>>
>> I'll discuss with Howard, and see.  I hate seeing more options added to
>> slapd, but it may be the only option (no pun intended!) for this scenario.
>> ;)  The cloud was fairly nascent when this was all designed, so this case
>> wasn't really a consideration at that point in time.  If you were to pass in
>> the server ID, I think you could get rid of the multi-valued serverID bit
>> entirely, since every server would know who it was already.
> 
> I think you would still need that. Maybe I have missed something else
> here, but how does slapd avoid making a syncrepl connection to itself
> in the replicated-config scenario?

Usually I avoid this by config management vars.

> I was assuming that it just ignores
> syncrepl clauses where the provider URL matches its own hostname.
> If it is done by IP then more thought will be required.

My recommendation for multi-homed scenarios and/or rapidly changing IP
addresses is simply to avoid multi-valued serverID config.

> The multi-valued attribute is neat in the case of servers with simple
> IP<->hostname mappings so we would probably want to keep it for that.

It's also useful for a client to ask slapd for the current effective
serverID of a provider replica.

E.g. in Æ-DIR there's a script which sends notification about added user
entry with further information to the user. If I have n replicas I don't
want n e-mails to be sent out. So the script searches new entries
created locally by using serverID part in entryCSN assertion value.
Would be cumbersome to sort out the right serverID from a multi-valued
serverID set.

So yet another idea for a feature request was having a separate
operational attribute in cn=config or cn=monitor reporting the current
effective serverID.

Ciao, Michael.



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Using virtual IP and N-way mutlimaster mode

[Quanah Gibson-Mount]

--On Wednesday, January 24, 2018 5:39 PM + Andrew Findlay 
 wrote:




In scenario #1, lmdb refuses to set the maxsize to something less than
the size of the DB.  It will make it equal to the size of the DB
(Leading to scenario #2)


Is 'size of the DB' the current length of the DB file, or the amount of
disk space actually used by it?


It's the amount of disk space used (DB+freespace table, etc)


In scenario #2, the server will not accept any new growth (write ops
will be rejected).  Monitoring should alert you to that, but the system
will recover once you adjust the maxsize to something useful.

In scenario #3, the maxsize is increased.

I've not seen any load related issues for such an operation, so it's not
clear to me what you mean by that, either.


Just to be clear on this, is it OK to change maxsize in the config file
while slapd is down or does this only work with LDAP-based config changes?


If you change the maxsize while slapd is down, it won't get applied until 
you start slapd.  If you're using cn=config, and change it while slapd's 
running, the change is immediate.  My deployments used cn=config, and the 
script would just modify the maxsize while slapd was running.



The other problem is definitely interesting, and I'm not clear on a good
solution for it.


How about adding a flag to slapd to specify the server ID so that it can
go the other way through the config, convert ID to FQDN, and drop that
FQDN from the set of replication sources?


I'll discuss with Howard, and see.  I hate seeing more options added to 
slapd, but it may be the only option (no pun intended!) for this scenario. 
;)  The cloud was fairly nascent when this was all designed, so this case 
wasn't really a consideration at that point in time.  If you were to pass 
in the server ID, I think you could get rid of the multi-valued serverID 
bit entirely, since every server would know who it was already.


--Quanah

--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

Re: Using virtual IP and N-way mutlimaster mode

[Andrew Findlay]

On Wed, Jan 24, 2018 at 09:27:20AM -0800, Quanah Gibson-Mount wrote:

> Please ellaborate on what is risky (or load inducing) about changing the
> maxsize.

It is entirely my assumption - wrong it would appear, which is good news!
I was thinking of this as something like grow/shrink ops on filesystems.

>  I've had a script for years that has automatically adjusted the
> maxsize as necessary based on the size of the DB vs maxsize, etc, via
> cn=config.  This script has been deployed on servers throughout the world
> via my former job @ Zimbra. Changing the maxsize has never been a problem.
> 
> You have 3 basic scenarios:
> 
> #1: You decrease the maxsize to < size of the DB
> #2: You set the maxsize = to the size of the DB
> #3: You increase the maxsize > size of the DB
> 
> In scenario #1, lmdb refuses to set the maxsize to something less than the
> size of the DB.  It will make it equal to the size of the DB (Leading to
> scenario #2)

Is 'size of the DB' the current length of the DB file, or the amount of disk
space actually used by it?

> In scenario #2, the server will not accept any new growth (write ops will be
> rejected).  Monitoring should alert you to that, but the system will recover
> once you adjust the maxsize to something useful.
> 
> In scenario #3, the maxsize is increased.
> 
> I've not seen any load related issues for such an operation, so it's not
> clear to me what you mean by that, either.

Just to be clear on this, is it OK to change maxsize in the config file while
slapd is down or does this only work with LDAP-based config changes?

> The other problem is definitely interesting, and I'm not clear on a good
> solution for it.

How about adding a flag to slapd to specify the server ID so that it can go the
other way through the config, convert ID to FQDN, and drop that FQDN from the
set of replication sources?

Andrew
-- 
---
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/+44 1628 782565 |
---

Re: Using virtual IP and N-way mutlimaster mode

[Andrew Findlay]

On Wed, Jan 24, 2018 at 06:37:43AM -0800, Quanah Gibson-Mount wrote:

> You can change the MDB maxsize at any time, including while slapd is
> running.  So that scenario at least doesn't compute. ;)

I would still rather do such risky / load-inducing changes on one replica at a
time. Anyway, running with fully-replicated config makes it even more important
to have a good solution to the problem I described.

Andrew
-- 
---
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/+44 1628 782565 |
---

Re: Using virtual IP and N-way mutlimaster mode

[Quanah Gibson-Mount]

--On Wednesday, January 24, 2018 3:06 PM + Andrew Findlay 
 wrote:



On Wed, Jan 24, 2018 at 06:37:43AM -0800, Quanah Gibson-Mount wrote:


You can change the MDB maxsize at any time, including while slapd is
running.  So that scenario at least doesn't compute. ;)


I would still rather do such risky / load-inducing changes on one replica
at a time. Anyway, running with fully-replicated config makes it even
more important to have a good solution to the problem I described.


Please ellaborate on what is risky (or load inducing) about changing the 
maxsize.  I've had a script for years that has automatically adjusted the 
maxsize as necessary based on the size of the DB vs maxsize, etc, via 
cn=config.  This script has been deployed on servers throughout the world 
via my former job @ Zimbra. Changing the maxsize has never been a problem.


You have 3 basic scenarios:

#1: You decrease the maxsize to < size of the DB
#2: You set the maxsize = to the size of the DB
#3: You increase the maxsize > size of the DB

In scenario #1, lmdb refuses to set the maxsize to something less than the 
size of the DB.  It will make it equal to the size of the DB (Leading to 
scenario #2)


In scenario #2, the server will not accept any new growth (write ops will 
be rejected).  Monitoring should alert you to that, but the system will 
recover once you adjust the maxsize to something useful.


In scenario #3, the maxsize is increased.

I've not seen any load related issues for such an operation, so it's not 
clear to me what you mean by that, either.



The other problem is definitely interesting, and I'm not clear on a good 
solution for it.


--Quanah

--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

Re: Using virtual IP and N-way mutlimaster mode

[Quanah Gibson-Mount]

--On Wednesday, January 24, 2018 11:59 AM + Andrew Findlay 
 wrote:



What is best practice here? I am leaning towards *not* replicating config
because it makes some updates really hard (think changing the max size of
an MDB database without stopping all the servers at the same time). I
still like the idea that the config should be identical on all servers
during normal operation though.


You can change the MDB maxsize at any time, including while slapd is 
running.  So that scenario at least doesn't compute. ;)


--Quanah


--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

Re: publisher migration or upgrade

[Quanah Gibson-Mount]

--On Sunday, January 21, 2018 5:30 PM -0500 Scott Bickford 
 wrote:



I have a RHEL 6 OpenLDAP 2.4.40 publisher with three subscribers in
production of the same version.  In development i also have a RHEL 6
OpenLDAP 2.4.40 and one subscriber of the same version.

I got a RHEL 7 OpenLDAP 2.4.44 subscriber replicating in development. 
I was able to load via slapadd the subscriber in about 30 minutes from an
overnight slapcat ldif.files from the publisher.

How can I upgrade the publisher without too much down time and ultimately
do the same in production?  I want to also keep the same host names.


Hi Scott,

You've left out a bit of detail (such as the backend in use) or what flags 
you used with slapadd (i.e., did you use -q for a quick slapadd? ) etc, so 
it's not easy to provide any answers to your questions.


--Quaanh



--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

Re: Using virtual IP and N-way mutlimaster mode

[Andrew Findlay]

On Tue, Jan 16, 2018 at 10:24:42AM +0400, Jephte Clain wrote:

> all nodes run with -h ldap:// and I have as many olcServerID as defined nodes
> my cn=config is also replicated  between all nodes
> 
> for example, I have:

> olcServerID: 1 ldap://ldapm3.univ.run/
> olcServerID: 2 ldap://ldapm4.univ.run/

> olcSyncrepl: {0}... rid=0 provider="ldap://ldapm3.univ.run/; ...
> olcSyncrepl: {1}... rid=3 provider="ldap://ldapm4.univ.run/; ...

There is an important variation of this problem when running cloud
machines in services like AWS. The actual machines have IPv4 addresses
in private address space - usually 10.x.x.x - but if you need to access
them from outside their own availability zone (datacentre) you must use
a different (public) IP address. The machines usually have hostnames of
the form ip-10-x-x-x and there is no way to resolve those globally.

So, we might have one server in each of three zones and need to set up
MMR between them:

Zone A:
Server IP: 10.21.0.1
Public IP: 134.82.17.63
Default hostname: ip-10-21-0-1

Zone B:
Server IP: 10.22.0.71
Public IP: 193.17.65.123
Default hostname: ip-10-22-0-71

Zone C:
Server IP: 10.35.0.5
Public IP: 7.45.81.52
Default hostname: ip-10-35-0-5

Not too much of a problem if each server has a separate config, but if
we want replicated configs it gets harder.

The first problem is that the private IP ranges used by the three zones
may overlap. Indeed, there are cases where the three servers could have
identical local IPs (and thus identical default hostnames).

Even without that, we have to cope with the fact that the servers are on
private addresses but have to contact each other using public addresses.

One option would be to set new hostnames on the servers and use those
to key into the appropriate config values. The hostnames would need to be
resolvable to public IP addresses so that the MMR config would work properly.
Does anything check that the hostname actually resolves to an IP address
bound to the server I wonder? Even if OpenLDAP is OK with that, it might
break something else running on the machine.

Another option would be to use the new hostnames in the -h URL list, but as has
already been mentioned this will fail because the server cannot bind to the
public IP.

What is best practice here? I am leaning towards *not* replicating config
because it makes some updates really hard (think changing the max size of
an MDB database without stopping all the servers at the same time). I still
like the idea that the config should be identical on all servers during
normal operation though.

[ Note that I am not considering the OP's virtual IP problem here, as
load-balancing/failover in this environment would be done separately ]

Andrew
-- 
---
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/+44 1628 782565 |
---