Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 26/06/15 14:15, Petr Vobornik wrote: On 06/17/2015 04:11 PM, Petr Vobornik wrote: On 06/17/2015 02:15 PM, Ludwig Krispenz wrote: On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch 878 topology: check topology in ipa-replica-manage del we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a degree of connectivity, eg to find single points of failure. In a topology A -- B -- C -- D, if B or C are down (temporariliy) the topology is disconnected. If extending to A -- B -- C -- D -- A one server con be taken offline, so a brute force would be to check for each server if it could be removed The original POC(attached) of the graph traversal did such brute force check(only one server removed at a time). In other words, it's easy. Computing indegree and outdegree of each node is easy as well. Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 Rebased patch attached. No new check was implemented. Works for me, ACK. -- David Kupka -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/29/2015 03:33 PM, David Kupka wrote: On 26/06/15 14:15, Petr Vobornik wrote: On 06/17/2015 04:11 PM, Petr Vobornik wrote: On 06/17/2015 02:15 PM, Ludwig Krispenz wrote: On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch 878 topology: check topology in ipa-replica-manage del we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a degree of connectivity, eg to find single points of failure. In a topology A -- B -- C -- D, if B or C are down (temporariliy) the topology is disconnected. If extending to A -- B -- C -- D -- A one server con be taken offline, so a brute force would be to check for each server if it could be removed The original POC(attached) of the graph traversal did such brute force check(only one server removed at a time). In other words, it's easy. Computing indegree and outdegree of each node is easy as well. Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 Rebased patch attached. No new check was implemented. Works for me, ACK. Pushed to master: 5397150979a474f6df82e6df5287e1cc678a3479 -- Petr Vobornik -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/17/2015 04:11 PM, Petr Vobornik wrote:
On 06/17/2015 02:15 PM, Ludwig Krispenz wrote:
On 06/17/2015 02:04 PM, Petr Vobornik wrote:
With patch 878 topology: check topology in ipa-replica-manage del
we can use the same logic for POC of
ipa topologysuffix-verify
command.
Checks done:
1. check if the topology is not disconnected. In other words if
there are replication paths between all servers.
2. check if servers don't have more than a recommended number of
replication agreements (which was set to 4)
I'm not sure what else we want to test but these two seemed as low
hanging fruit.
don't know how hard it is, but I had thought of calculating something
like a degree of connectivity, eg to find single points of failure.
In a topology A -- B -- C -- D, if B or C are down (temporariliy)
the topology is disconnected. If extending to
A -- B -- C -- D -- A one server con be taken offline, so a
brute force would be to check for each server if it could be removed
The original POC(attached) of the graph traversal did such brute force
check(only one server removed at a time). In other words, it's easy.
Computing indegree and outdegree of each node is easy as well.
Additional checks can be also added later.
https://fedorahosted.org/freeipa/ticket/4302
Rebased patch attached. No new check was implemented.
--
Petr Vobornik
From 4fe4009263d8890cd5872e7a4f19923bdf3351d6 Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Wed, 17 Jun 2015 13:50:32 +0200
Subject: [PATCH] Verify replication topology for a suffix
Checks done:
1. check if the topology is not disconnected. In other words if
there are replication paths between all servers.
2. check if servers don't have more than a recommended number of
replication agreements(4)
https://fedorahosted.org/freeipa/ticket/4302
---
API.txt| 5 +++
VERSION| 4 +--
ipalib/constants.py| 4 +++
ipalib/plugins/topology.py | 83 ++
4 files changed, 94 insertions(+), 2 deletions(-)
diff --git a/API.txt b/API.txt
index 3bcb3bdd24ada4e513f6263fc32a2953c18fc142..bccebe55da8a785cbb6ca782904d7523c4a9322f 100644
--- a/API.txt
+++ b/API.txt
@@ -4911,6 +4911,11 @@ option: Str('version?', exclude='webui')
output: Entry('result', type 'dict', Gettext('A dictionary representing an LDAP entry', domain='ipa', localedir=None))
output: Output('summary', (type 'unicode', type 'NoneType'), None)
output: PrimaryKey('value', None, None)
+command: topologysuffix_verify
+args: 1,1,1
+arg: Str('cn', attribute=True, cli_name='name', multivalue=False, primary_key=True, query=True, required=True)
+option: Str('version?', exclude='webui')
+output: Output('result', None, None)
command: trust_add
args: 1,13,3
arg: Str('cn', attribute=True, cli_name='realm', multivalue=False, primary_key=True, required=True)
diff --git a/VERSION b/VERSION
index 224d34925685c8ecb6f2db3672d34c40621dc9dc..2f884ff73afad57f35f06ce279add5c078073353 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
# #
IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=135
-# Last change: jcholast - User life cycle: Make user-del flags CLI-specific
+IPA_API_VERSION_MINOR=136
+# Last change: pvoborni: add topologysuffix-verify command
diff --git a/ipalib/constants.py b/ipalib/constants.py
index 330f9df74e604d9875a7a9624312ea8944d5..a062505c349436332d430af4fd29c76d20c85343 100644
--- a/ipalib/constants.py
+++ b/ipalib/constants.py
@@ -170,6 +170,10 @@ DEFAULT_CONFIG = (
# KRA plugin
('kra_host', FQDN), # Set in Env._finalize_core()
+# Topology plugin
+('recommended_max_agmts', 4), # Recommended maximum number of replication
+ # agreements
+
# Special CLI:
('prompt_all', False),
('interactive', True),
diff --git a/ipalib/plugins/topology.py b/ipalib/plugins/topology.py
index 494d3bb0a564e5c8ef3d7c2af50dbf1e83a36e1f..49060d672b6522277014b0b9c1e0ecb92e091077 100644
--- a/ipalib/plugins/topology.py
+++ b/ipalib/plugins/topology.py
@@ -10,6 +10,7 @@ from ipalib.plugins.baseldap import (
LDAPRetrieve)
from ipalib import _, ngettext
from ipalib import output
+from ipalib.util import create_topology_graph, get_topology_connection_errors
from ipapython.dn import DN
@@ -401,3 +402,85 @@ class topologysuffix_mod(LDAPUpdate):
@register()
class topologysuffix_show(LDAPRetrieve):
__doc__ = _('Show managed suffix.')
+
+
+@register()
+class topologysuffix_verify(LDAPQuery):
+__doc__ = _('''
+Verify replication topology for suffix.
+
+Checks done:
+ 1. check if a topology is not disconnected. In other words if there are
+ replication paths between all servers.
+ 2. check if servers don't have more than the recommended number of
+ replication agreements
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch 878 topology: check topology in ipa-replica-manage del we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a degree of connectivity, eg to find single points of failure. In a topology A -- B -- C -- D, if B or C are down (temporariliy) the topology is disconnected. If extending to A -- B -- C -- D -- A one server con be taken offline, so a brute force would be to check for each server if it could be removed Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/17/2015 02:15 PM, Ludwig Krispenz wrote:
On 06/17/2015 02:04 PM, Petr Vobornik wrote:
With patch 878 topology: check topology in ipa-replica-manage del
we can use the same logic for POC of
ipa topologysuffix-verify
command.
Checks done:
1. check if the topology is not disconnected. In other words if
there are replication paths between all servers.
2. check if servers don't have more than a recommended number of
replication agreements (which was set to 4)
I'm not sure what else we want to test but these two seemed as low
hanging fruit.
don't know how hard it is, but I had thought of calculating something
like a degree of connectivity, eg to find single points of failure.
In a topology A -- B -- C -- D, if B or C are down (temporariliy)
the topology is disconnected. If extending to
A -- B -- C -- D -- A one server con be taken offline, so a
brute force would be to check for each server if it could be removed
The original POC(attached) of the graph traversal did such brute force
check(only one server removed at a time). In other words, it's easy.
Computing indegree and outdegree of each node is easy as well.
Additional checks can be also added later.
https://fedorahosted.org/freeipa/ticket/4302
--
Petr Vobornik
#!/usr/bin/python
# Python Breath First Search
#
# Intendted for FreeIPA topology check
# structure of segment:
#
# edges:
#
# topologysegment
# - cn
# - iparepltoposegmentrightnode
# - iparepltoposegmentdirection
# - iparepltoposegmentleftnode
#
#
# vertices:
# masters
# - cn
class Graph():
Simple graph structure with vertices, edges and adjectency list
vertices = set()
edges = []
adj = dict()
def add_vertex(self, vertex):
self.vertices.add(vertex)
self.adj[vertex] = []
def add_edge(self, left, right):
self.edges.append((left, right))
self.adj[left].append(right)
def remove_vertex(self, vertex):
self.vertices.remove(vertex)
# delete adjacencies
del self.adj[vertex]
for key, adj in self.adj.iteritems():
adj[:] = [v for v in adj if v != vertex]
# delete edges
edges = [e for e in self.edges if e[0] != vertex and e[1] != vertex]
self.edges[:] = edges
def bfs(graph, start=None):
if not start:
start = list(graph.vertices)[0]
visited = set()
queue = [start]
while queue:
vertex = queue.pop(0)
if vertex not in visited:
visited.add(vertex)
queue.extend(set(graph.adj.get(vertex, [])) - visited)
return visited
def make_graph(servers, segments):
graph = Graph()
for s in servers:
graph.add_vertex(s)
for s in segments:
direction = s[2]
if direction == 'both':
graph.add_edge(s[0], s[1])
graph.add_edge(s[1], s[0])
if direction == 'left-right':
graph.add_edge(s[0], s[1])
if direction == 'right-left':
graph.add_edge(s[1], s[0])
return graph
def print_results(errors):
if not errors:
print Initial topology is in order. All servers can replicate
else:
print Initial topology has some errors
print_errors(errors)
def print_results_removed(removed, errors):
if not errors:
print removed + is safe to remove
else:
print removed + is not safe to remove
print_errors(errors)
def print_errors(errors):
for e in errors:
print e[0] + can't contact: + ', '.join(e[2])
def iterate_start(graph):
servers = list(graph.vertices)
servers.sort()
e = []
for s in servers:
visited = bfs(graph, s)
not_visited = set(servers) - visited
if not_visited:
e.append((s, visited, not_visited))
return e
def test():
servers = {'a', 'b', 'c', 'd'}
# left, right, direction
segments = [
('a', 'b', 'both'),
('b', 'd', 'both'),
('a', 'd', 'right-left'),
('c', 'd', 'right-left'),
('c', 'd', 'left-right'),
]
g = make_graph(servers, segments)
visited = bfs(g)
not_visited = servers - visited
print
print = all servers ===
errors = iterate_start(g)
print_results(errors)
for s in servers:
g = make_graph(servers, segments)
g.remove_vertex(s)
print
print = removing: + s + ===
errors = iterate_start(g)
print_results_removed(s, errors)
test()
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
