Hi everybody,
Current implementation of topology plugin (including patch 878 from
Petr) allows the deletion of the central node in the star topology.
I had the following topology:
vm056 vm036
\ / |
vm175 |
/ \ |
vm127 vm244
I was able to remove node vm175 from node vm244:
[17:54:48]ofayans@vm-244:~]$ ipa-replica-manage del
vm-175.idm.lab.eng.brq.redhat.com
Topology after removal of vm-175.idm.lab.eng.brq.redhat.com will be
disconnected:
Server vm-036.idm.lab.eng.brq.redhat.com can't contact servers:
vm-056.idm.lab.eng.brq.redhat.com, vm-127.idm.lab.eng.brq.redhat.com
Server vm-056.idm.lab.eng.brq.redhat.com can't contact servers:
vm-244.idm.lab.eng.brq.redhat.com,
vm-036.idm.lab.eng.brq.redhat.com, vm-127.idm.lab.eng.brq.redhat.com
Server vm-127.idm.lab.eng.brq.redhat.com can't contact servers:
vm-244.idm.lab.eng.brq.redhat.com,
vm-056.idm.lab.eng.brq.redhat.com, vm-036.idm.lab.eng.brq.redhat.com
Server vm-244.idm.lab.eng.brq.redhat.com can't contact servers:
vm-056.idm.lab.eng.brq.redhat.com, vm-127.idm.lab.eng.brq.redhat.com
Continue to delete? [no]: yes
Waiting for removal of replication agreements
unexpected error: limits exceeded for this query
I would expect this operation to delete 4 replication agreements on
all nodes:
vm056 - vm175
vm127 - vm175
vm244 - vm175
vm036 - vm175
However an arbitrary set of replication agreements was deleted on
each node leading to total infrastructure inconsistency:
===============================================================
vm056**thought the topology was as follows:
vm056 vm036
/ |
vm175 |
/ \ |
vm127 vm244
[10:28:55]ofayans@vm-056:~]$ ipa topologysegment-find realm
------------------
4 segments matched
------------------
Segment name: 036-to-244
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-036.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-127.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-127.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-175.idm.lab.eng.brq.redhat.com-to-vm-244.idm.lab.eng.brq.redhat.com
Left node: vm-175.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
----------------------------
Number of entries returned 4
----------------------------
===============================================================
both vm036**vm244 thought the topology was as follows:
vm056 vm036
\ |
vm175 |
/ |
vm127 vm244
[10:26:23]ofayans@vm-036:~]$ ipa topologysegment-find
Suffix name: realm
------------------
3 segments matched
------------------
Segment name: 036-to-244
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-056.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-127.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-127.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
----------------------------
Number of entries returned 3
----------------------------
===============================================================
**vm127 thought the topology was as follows:
vm056 vm036
\ / |
vm175 |
\ |
vm127 vm244
[10:31:08]ofayans@vm-127:~]$ ipa topologysegment-find realm
------------------
4 segments matched
------------------
Segment name: 036-to-244
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-036.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-056.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-175.idm.lab.eng.brq.redhat.com-to-vm-244.idm.lab.eng.brq.redhat.com
Left node: vm-175.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
----------------------------
Number of entries returned 4
----------------------------
If I, for example, add a segment connecting vm127 and vm244, these
two nodes will not synchronize the topology info:
[10:51:03]ofayans@vm-127:~]$ ipa topologysegment-add realm
127-to-244 --leftnode=vm-127.idm.lab.eng.brq.redhat.com
--rightnode=vm-244.idm.lab.eng.brq.redhat.com --direction=both
--------------------------
Added segment "127-to-244"
--------------------------
Segment name: 127-to-244
Left node: vm-127.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
[10:53:33]ofayans@vm-127:~]$ ipa topologysegment-find realm
------------------
5 segments matched
------------------
Segment name: 036-to-244
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name: 127-to-244
Left node: vm-127.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-036.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-056.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-175.idm.lab.eng.brq.redhat.com-to-vm-244.idm.lab.eng.brq.redhat.com
Left node: vm-175.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
----------------------------
Number of entries returned 5
----------------------------
[10:54:02]ofayans@vm-127:~]$
=============================================================
[10:49:38]ofayans@vm-244:~]$ ipa topologysegment-find realm
------------------
3 segments matched
------------------
Segment name: 036-to-244
Left node: vm-036.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name: 127-to-244
Left node: vm-127.idm.lab.eng.brq.redhat.com
Right node: vm-244.idm.lab.eng.brq.redhat.com
Connectivity: both
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com
Left node: vm-056.idm.lab.eng.brq.redhat.com
Right node: vm-175.idm.lab.eng.brq.redhat.com
Connectivity: both
----------------------------
Number of entries returned 3
----------------------------
[10:56:34]ofayans@vm-244:~]$
Conclusion:
We either should completely prohibit the removal of the middle nodes
(I mean, nodes that hide another active nodes),
or at the removal stage first recalculate the resulting topology and
send it to all nodes before actual removal.
--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.