Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/29/2015 03:33 PM, David Kupka wrote: On 26/06/15 14:15, Petr Vobornik wrote: On 06/17/2015 04:11 PM, Petr Vobornik wrote: On 06/17/2015 02:15 PM, Ludwig Krispenz wrote: On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch "878 topology: check topology in ipa-replica-manage del" we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a "degree of connectivity", eg to find single points of failure. In a topology A <--> B <--> C <--> D, if B or C are down (temporariliy) the topology is disconnected. If extending to A <--> B <--> C <--> D <--> A one server con be taken offline, so a brute force would be to check for each server if it could be removed The original POC(attached) of the graph traversal did such brute force check(only one server removed at a time). In other words, it's easy. Computing indegree and outdegree of each node is easy as well. Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 Rebased patch attached. No new check was implemented. Works for me, ACK. Pushed to master: 5397150979a474f6df82e6df5287e1cc678a3479 -- Petr Vobornik -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 26/06/15 14:15, Petr Vobornik wrote: On 06/17/2015 04:11 PM, Petr Vobornik wrote: On 06/17/2015 02:15 PM, Ludwig Krispenz wrote: On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch "878 topology: check topology in ipa-replica-manage del" we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a "degree of connectivity", eg to find single points of failure. In a topology A <--> B <--> C <--> D, if B or C are down (temporariliy) the topology is disconnected. If extending to A <--> B <--> C <--> D <--> A one server con be taken offline, so a brute force would be to check for each server if it could be removed The original POC(attached) of the graph traversal did such brute force check(only one server removed at a time). In other words, it's easy. Computing indegree and outdegree of each node is easy as well. Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 Rebased patch attached. No new check was implemented. Works for me, ACK. -- David Kupka -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/17/2015 04:11 PM, Petr Vobornik wrote: On 06/17/2015 02:15 PM, Ludwig Krispenz wrote: On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch "878 topology: check topology in ipa-replica-manage del" we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a "degree of connectivity", eg to find single points of failure. In a topology A <--> B <--> C <--> D, if B or C are down (temporariliy) the topology is disconnected. If extending to A <--> B <--> C <--> D <--> A one server con be taken offline, so a brute force would be to check for each server if it could be removed The original POC(attached) of the graph traversal did such brute force check(only one server removed at a time). In other words, it's easy. Computing indegree and outdegree of each node is easy as well. Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 Rebased patch attached. No new check was implemented. -- Petr Vobornik From 4fe4009263d8890cd5872e7a4f19923bdf3351d6 Mon Sep 17 00:00:00 2001 From: Petr Vobornik Date: Wed, 17 Jun 2015 13:50:32 +0200 Subject: [PATCH] Verify replication topology for a suffix Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements(4) https://fedorahosted.org/freeipa/ticket/4302 --- API.txt| 5 +++ VERSION| 4 +-- ipalib/constants.py| 4 +++ ipalib/plugins/topology.py | 83 ++ 4 files changed, 94 insertions(+), 2 deletions(-) diff --git a/API.txt b/API.txt index 3bcb3bdd24ada4e513f6263fc32a2953c18fc142..bccebe55da8a785cbb6ca782904d7523c4a9322f 100644 --- a/API.txt +++ b/API.txt @@ -4911,6 +4911,11 @@ option: Str('version?', exclude='webui') output: Entry('result', , Gettext('A dictionary representing an LDAP entry', domain='ipa', localedir=None)) output: Output('summary', (, ), None) output: PrimaryKey('value', None, None) +command: topologysuffix_verify +args: 1,1,1 +arg: Str('cn', attribute=True, cli_name='name', multivalue=False, primary_key=True, query=True, required=True) +option: Str('version?', exclude='webui') +output: Output('result', None, None) command: trust_add args: 1,13,3 arg: Str('cn', attribute=True, cli_name='realm', multivalue=False, primary_key=True, required=True) diff --git a/VERSION b/VERSION index 224d34925685c8ecb6f2db3672d34c40621dc9dc..2f884ff73afad57f35f06ce279add5c078073353 100644 --- a/VERSION +++ b/VERSION @@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412 # # IPA_API_VERSION_MAJOR=2 -IPA_API_VERSION_MINOR=135 -# Last change: jcholast - User life cycle: Make user-del flags CLI-specific +IPA_API_VERSION_MINOR=136 +# Last change: pvoborni: add topologysuffix-verify command diff --git a/ipalib/constants.py b/ipalib/constants.py index 330f9df74e604d9875a7a9624312ea8944d5..a062505c349436332d430af4fd29c76d20c85343 100644 --- a/ipalib/constants.py +++ b/ipalib/constants.py @@ -170,6 +170,10 @@ DEFAULT_CONFIG = ( # KRA plugin ('kra_host', FQDN), # Set in Env._finalize_core() +# Topology plugin +('recommended_max_agmts', 4), # Recommended maximum number of replication + # agreements + # Special CLI: ('prompt_all', False), ('interactive', True), diff --git a/ipalib/plugins/topology.py b/ipalib/plugins/topology.py index 494d3bb0a564e5c8ef3d7c2af50dbf1e83a36e1f..49060d672b6522277014b0b9c1e0ecb92e091077 100644 --- a/ipalib/plugins/topology.py +++ b/ipalib/plugins/topology.py @@ -10,6 +10,7 @@ from ipalib.plugins.baseldap import ( LDAPRetrieve) from ipalib import _, ngettext from ipalib import output +from ipalib.util import create_topology_graph, get_topology_connection_errors from ipapython.dn import DN @@ -401,3 +402,85 @@ class topologysuffix_mod(LDAPUpdate): @register() class topologysuffix_show(LDAPRetrieve): __doc__ = _('Show managed suffix.') + + +@register() +class topologysuffix_verify(LDAPQuery): +__doc__ = _(''' +Verify replication topology for suffix. + +Checks done: + 1. check if a topology is not disconnected. In other words if there are + replication paths between all servers. + 2. check if servers don't have more than the recommended number of + replication agreements +''') + +def execute(self, *keys, **opti
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/17/2015 02:15 PM, Ludwig Krispenz wrote: On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch "878 topology: check topology in ipa-replica-manage del" we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a "degree of connectivity", eg to find single points of failure. In a topology A <--> B <--> C <--> D, if B or C are down (temporariliy) the topology is disconnected. If extending to A <--> B <--> C <--> D <--> A one server con be taken offline, so a brute force would be to check for each server if it could be removed The original POC(attached) of the graph traversal did such brute force check(only one server removed at a time). In other words, it's easy. Computing indegree and outdegree of each node is easy as well. Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 -- Petr Vobornik #!/usr/bin/python # Python Breath First Search # # Intendted for FreeIPA topology check # structure of segment: # # edges: # # topologysegment # - cn # - iparepltoposegmentrightnode # - iparepltoposegmentdirection # - iparepltoposegmentleftnode # # # vertices: # masters # - cn class Graph(): """ Simple graph structure with vertices, edges and adjectency list """ vertices = set() edges = [] adj = dict() def add_vertex(self, vertex): self.vertices.add(vertex) self.adj[vertex] = [] def add_edge(self, left, right): self.edges.append((left, right)) self.adj[left].append(right) def remove_vertex(self, vertex): self.vertices.remove(vertex) # delete adjacencies del self.adj[vertex] for key, adj in self.adj.iteritems(): adj[:] = [v for v in adj if v != vertex] # delete edges edges = [e for e in self.edges if e[0] != vertex and e[1] != vertex] self.edges[:] = edges def bfs(graph, start=None): if not start: start = list(graph.vertices)[0] visited = set() queue = [start] while queue: vertex = queue.pop(0) if vertex not in visited: visited.add(vertex) queue.extend(set(graph.adj.get(vertex, [])) - visited) return visited def make_graph(servers, segments): graph = Graph() for s in servers: graph.add_vertex(s) for s in segments: direction = s[2] if direction == 'both': graph.add_edge(s[0], s[1]) graph.add_edge(s[1], s[0]) if direction == 'left-right': graph.add_edge(s[0], s[1]) if direction == 'right-left': graph.add_edge(s[1], s[0]) return graph def print_results(errors): if not errors: print "Initial topology is in order. All servers can replicate" else: print "Initial topology has some errors" print_errors(errors) def print_results_removed(removed, errors): if not errors: print removed + " is safe to remove" else: print removed + " is not safe to remove" print_errors(errors) def print_errors(errors): for e in errors: print e[0] + " can't contact: " + ', '.join(e[2]) def iterate_start(graph): servers = list(graph.vertices) servers.sort() e = [] for s in servers: visited = bfs(graph, s) not_visited = set(servers) - visited if not_visited: e.append((s, visited, not_visited)) return e def test(): servers = {'a', 'b', 'c', 'd'} # left, right, direction segments = [ ('a', 'b', 'both'), ('b', 'd', 'both'), ('a', 'd', 'right-left'), ('c', 'd', 'right-left'), ('c', 'd', 'left-right'), ] g = make_graph(servers, segments) visited = bfs(g) not_visited = servers - visited print "" print "= all servers ===" errors = iterate_start(g) print_results(errors) for s in servers: g = make_graph(servers, segments) g.remove_vertex(s) print "" print "= removing: " + s + " ===" errors = iterate_start(g) print_results_removed(s, errors) test() -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 879 Verify replication topology for a suffix
On 06/17/2015 02:04 PM, Petr Vobornik wrote: With patch "878 topology: check topology in ipa-replica-manage del" we can use the same logic for POC of ipa topologysuffix-verify command. Checks done: 1. check if the topology is not disconnected. In other words if there are replication paths between all servers. 2. check if servers don't have more than a recommended number of replication agreements (which was set to 4) I'm not sure what else we want to test but these two seemed as low hanging fruit. don't know how hard it is, but I had thought of calculating something like a "degree of connectivity", eg to find single points of failure. In a topology A <--> B <--> C <--> D, if B or C are down (temporariliy) the topology is disconnected. If extending to A <--> B <--> C <--> D <--> A one server con be taken offline, so a brute force would be to check for each server if it could be removed Additional checks can be also added later. https://fedorahosted.org/freeipa/ticket/4302 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code