Re: [Freeipa-devel] topology issues

Petr Vobornik Wed, 10 Jun 2015 05:52:44 -0700

On 06/10/2015 02:42 PM, thierry bordaz wrote:

On 06/10/2015 02:19 PM, Ludwig Krispenz wrote:


On 06/10/2015 02:13 PM, thierry bordaz wrote:

On 06/10/2015 10:51 AM, Ludwig Krispenz wrote:


On 06/10/2015 10:41 AM, Martin Basti wrote:

On 10/06/15 09:13, Ludwig Krispenz wrote:

Hi,

there seems to be somethin going wrong in the code to delete the
services.

The code is:

        # delete master entry with all active services
        try:
            dn = DN(('cn', replica), ('cn', 'masters'), ('cn',
'ipa'),
                    ('cn', 'etc'), self.suffix)
            entries = self.conn.get_entries(dn, ldap.SCOPE_SUBTREE)
            if entries:
                entries.sort(key=lambda x: len(x.dn), reverse=True)
                for entry in entries:
                    self.conn.delete_entry(entry)
        except errors.NotFound:
            pass
        except Exception, e:
            if not force:
                raise e
            elif not err:
                err = e

In the access log we see:


[09/Jun/2015:08:32:42 -0400] conn=150 op=52 SRCH
base="cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
scope=2 filter="(objectClass=*)" attrs=ALL
[09/Jun/2015:08:32:42 -0400] conn=150 op=52 RESULT err=0 tag=101
nentries=8 etime=0 notes=U

this was the get_entries, it returns 8 entries

[09/Jun/2015:08:32:42 -0400] conn=150 op=53 DEL
dn="cn=KDC,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"

[09/Jun/2015:08:32:42 -0400] conn=150 op=53 RESULT err=0 tag=107
nentries=0 etime=0 csn=5576dceb000600040000
[09/Jun/2015:08:32:42 -0400] conn=150 op=54 DEL
dn="cn=KPASSWD,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"

[09/Jun/2015:08:32:42 -0400] conn=150 op=54 RESULT err=0 tag=107
nentries=0 etime=0 csn=5576dceb000700040000
[09/Jun/2015:08:32:42 -0400] conn=150 op=55 DEL
dn="cn=MEMCACHE,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"

[09/Jun/2015:08:32:43 -0400] conn=150 op=55 RESULT err=0 tag=107
nentries=0 etime=1 csn=5576dcec000100040000

here it stops after deleting three entries, and it should do it in
reverse order of the dn length, but KDC is deleted before MEMCACHE


Something surprising is that according to the code

            # delete master entry with all active services
            try:
                dn = DN(('cn', replica), ('cn', 'masters'), ('cn',
    'ipa'),
                        ('cn', 'etc'), self.suffix)
                entries = self.conn.get_entries(dn, ldap.SCOPE_SUBTREE)
                if entries:
                    entries.sort(key=lambda x: len(x.dn), reverse=True)
                    for entry in entries:
                        self.conn.delete_entry(entry)
            except errors.NotFound:
                pass
            except Exception, e:
                if not force:
                    raise e
                elif not err:
                    err = e

            try:
                entry = *self.conn.get_entry*(
                    DN(('cn', 'ipa'), ('cn', 'etc'), self.suffix),
    ['aci'])

                sub = {'suffix': self.suffix, 'fqdn': replica}
            ...

We should see a search on cn=ipa,cn=etc,SUFFIX following the deletion
of those entries.
But the next op is an UNBIND.

yes, that is strange, maybe we hit an exception and the connection was
closed

With UNBIND logged, it looks like the closure is triggered by the CLI.
I agree it should be some exception but my understanding is that 'force'
was set. so when it started deleting entries any exception is caught and
we should do the following search.

Btw, the `ipa-replica-manage del <repl>` issues should have been handledby new version of tbabej's patch 329. Given that Tomas is on PTO, I'llupdate his patch to handle only 'connect' and 'disconnect' cases andcreate a new one for 'del'.

Is that the code executed by ipa-replica-manage ?

I think so, yes.

[09/Jun/2015:08:32:43 -0400] conn=150 op=56 UNBIND


Are there any ideas what is going on or how to debug it ?

Actually, the both DNs of KDC and MEMCACHE has the same length.
IPA implements own DN class, where length is the number of AVA/RDN
parts (mixed in code, but it means the 'cn=user' has length 1, and
'cn=user,cn=accounts' has length 2)

    def __len__(self):
        return len(self.rdns)

This reverse sort guarantees the child entries will be removed
before the parent entries.

thanks, then it is ok, but it does not explain why not all services
and the master were not deleted.


To debug, maybe print the entries from IPA code, before sort and
after sort might help.

yep, but so far only Oleg reprted this, and he's not here today, I
haven't reproduced the issue


Martin^2


On 06/09/2015 05:32 PM, Ludwig Krispenz wrote:

Hi Oleg,
thanks for access to your machine, the replication agreements are
still there - and that is expected since the server was not removed.

In the access log I see:

[09/Jun/2015:08:32:42 -0400] conn=150 op=52 SRCH
base="cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
scope=2 filter="(objectClass=*)" attrs=ALL
[09/Jun/2015:08:32:42 -0400] conn=150 op=52 RESULT err=0 tag=101
nentries=8 etime=0 notes=U
[09/Jun/2015:08:32:42 -0400] conn=150 op=53 DEL
dn="cn=KDC,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"

[09/Jun/2015:08:32:42 -0400] conn=150 op=53 RESULT err=0 tag=107
nentries=0 etime=0 csn=5576dceb000600040000
[09/Jun/2015:08:32:42 -0400] conn=150 op=54 DEL
dn="cn=KPASSWD,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"

[09/Jun/2015:08:32:42 -0400] conn=150 op=54 RESULT err=0 tag=107
nentries=0 etime=0 csn=5576dceb000700040000
[09/Jun/2015:08:32:42 -0400] conn=150 op=55 DEL
dn="cn=MEMCACHE,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"

[09/Jun/2015:08:32:43 -0400] conn=150 op=55 RESULT err=0 tag=107
nentries=0 etime=1 csn=5576dcec000100040000
[09/Jun/2015:08:32:43 -0400] conn=150 op=56 UNBIND

the search for cn=f22replica1.bagam.net,cn=masters,.... returns 8
entries, which then should be deleted, but only 3 ae deleted and the
cn=f22replica1.bagam.net,cn=masters,... entry is not deleted, so
the topology segments are not deleted, and the agreement is not
removed.

I don't know why ipa-replica-manage del does stop deleting
services and the master entry



On 06/09/2015 04:25 PM, Oleg Fayans wrote:



On 06/09/2015 04:19 PM, Ludwig Krispenz wrote:


On 06/09/2015 04:14 PM, Oleg Fayans wrote:



On 06/09/2015 04:04 PM, Ludwig Krispenz wrote:


On 06/09/2015 03:55 PM, Oleg Fayans wrote:

Hi everybody,

The current status of Topology plugin testing is as follows:

1. There is still no proper way of removing the replica.
Standard procedure using `ipa-replica-manage del` throws
"Server is unwilling to perform: Entry is managed by
topology plugin.Deletion not allowed.".

yes, that is for the first attempt to directly remove the
agreement, but when the server is removed the agreements
should be removed

We should probably think of less threatening error message in
this case. Just from reading the command output one might
conclude that replica removal failed.

The replication agreement though does get deleted,

then it is ok,

but the topology information does not get updated.

what do you mean, where do you check ? in the "remaining"
topology the shared tree should be updated, for the removed
replica it will not, but this should be uninstalled anyway

The problem here, is that the topology information does not
get updated on master as well.

could you be a bit more precise. what do you still see ? the
agreement will be only removed if the segment is removed, and
this should be reoplicated to all severs in the remaining
topology - if you don't disconnect it by removing the replica.
and what was the topology structure and which replica did you
remove, on which server did you remove it?

So,  Here is the results of the `topologysegment-find` command
before replica removal:
root@f22master:/var/log/dirsrv/slapd-BAGAM-NET]$ ipa
topologysegment-find
Suffix name: realm
------------------
2 segments matched
------------------
  Segment name: f22master.bagam.net-to-f22replica1.bagam.net
  Left node: f22master.bagam.net
  Right node: f22replica1.bagam.net
  Connectivity: both

  Segment name: f22master.bagam.net-to-f22replica2.bagam.net
  Left node: f22master.bagam.net
  Right node: f22replica2.bagam.net
  Connectivity: both
----------------------------
Number of entries returned 2
----------------------------
Then, after issuing `ipa-replica-manage-del f2replica1.bagam.net
--force` on the master, the same command on master still shows
exactly the same topology:

root@f22master:/var/log/dirsrv/slapd-BAGAM-NET]$ ipa
topologysegment-find
Suffix name: realm
------------------
2 segments matched
------------------
  Segment name: f22master.bagam.net-to-f22replica1.bagam.net
  Left node: f22master.bagam.net
  Right node: f22replica1.bagam.net
  Connectivity: both

  Segment name: f22master.bagam.net-to-f22replica2.bagam.net
  Left node: f22master.bagam.net
  Right node: f22replica2.bagam.net
  Connectivity: both
----------------------------
Number of entries returned 2
----------------------------

When I then issue `ipa topologysegment-del`, it fails due to
"ipa: ERROR: Server is unwilling to perform: Removal of
Segment disconnects topology.Deletion not allowed."

correct, you can only do it after removal of the server

I do not get it. Master still thinks it has the replica, it
displays it both in CLI using `ipa topologysegment-find` and
in the web-ui. (although it does not show it using `ipa
host-find`, which is correct), and there is no way to manually
make it change it's mind?


I tried to disable the segment first and then delete it, but
with the segment properly disabled, the attempt to delete it
raised a GSS error: "ipa: ERROR: Kerberos error: Kerberos
error: ('Unspecified GSS failure. Minor code may provide
more information', 851968)/('KDC returned error string:
PROCESS_TGS', -1765328324)/". I am not sure, where to search
for corresponding logs. The session transcript is attached.

2. The following is probably unrelated to the topology plugin:
I installed a replica with --setup-ca option. Then, on this
replica tried to prepare another replica:
-------------------------------------------------------------------------------------------------------------------------------------------------

root@f22replica2:/home/ofayans/f22]$ ipa-replica-prepare
--ip-address 192.168.122.141 f22replica3.bagam.net
Directory Manager (existing master) password:

Preparing replica for f22replica3.bagam.net from
f22replica2.bagam.net
Creating SSL certificate for the Directory Server
Certificate issuance failed
-------------------------------------------------------------------------------------------------------------------------------------------------

The corresponding line in the dirsrv log:
[09/Jun/2015:09:54:46 -0400] - Entry
"uid=admin,ou=people,o=ipaca" -- attribute "krbExtraData"
not allowed

--
Petr Vobornik

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] topology issues

Reply via email to