Re: [Freeipa-devel] Move replication topology to the shared tree
On Fri, 2014-06-06 at 09:03 -0400, Simo Sorce wrote: > On Fri, 2014-06-06 at 06:58 -0400, James wrote: > > On Mon, Jun 2, 2014 at 4:46 AM, Ludwig Krispenz wrote: > > > Ticket 4302 is a request for an enhancement: Move replication topology to > > > the shared tree > > > > > > One comment to add: > > > > It would be particularly useful if the interface used to set the > > topology is something sane that a single host can use to set its > > peers. Eg: > > > > host1$ ipa-topology-set-peers host2 > > host2$ ipa-topology-set-peers host3 > > host3$ ipa-topology-set-peers host4 > > host4$ ipa-topology-set-peers host1 # redundant, but okay... > > > > This way config management could define topologies in terms of algorithms. > > Eg: > > > > Ring topology: > > Order hostnames alphabetically. > > Peer with host name index of self + 1 > > (Example shown above) > > > > Star topology: > > Peer with every other host in list > > > > Pick two topology: > > Peer with each subsequent hosts in ordered list... > > > > And so on... > > If running the command changes the topology again, then that's great > > because changing the algorithm would re-arrange the graph :) > > > > Hope this made sense. > > Oh it does! > > But I am dreaming even bigger, Then I like you :) > my aim is to end up (in the long run) > with something like: > > ipa topology change --stellar host1 host2 host3 host4 host5 > > causes topology for those 5 servers only (ignores any other connection) > to become: > > host2 > | > host3 - host1 - host5 > | > host4 > > So any agreement between host1 and 2,3,4,5 get wiped and replaced with > the stellar topology. (agreements between 1,2,3,4,5 and 6,7,8,9 are not > affected in any way and stay). > > And later on you'd be able to say > ipa topology change --circular host1 host2 host3 host4 > > and you get: > > host1 - host2 > | | > host4 - host3 > > Note that if you previously did the stellar thing and you only have > these 5 servers the actual complete topology ends up being like this: > > host5 > | > host1 - host2 > | | > host4 - host3 > > As the agreement between host1 and host5 is not touched by the second > command. > > But this is in the far future IMO, we'll probably start by only > implementing: > > ipa topology set-peering host1 host2 > and > ipa topology break-peering host3 host4 Sounds good to me. > > Ie creating and breaking segments in the topology tree only between > pairs and storing/deleting the segment object in the shared tree. > > The reason is that the topology plugin will allow or disallow changes > based on whether you are breaking the graph, and it is much simpler to > do that if you change one object at a time. To do the nifty stuff above > we'd need some staging area where we describe all the changes and then > have a "commit" type change that would cause the topology plugin to do > all the calculations and move stuff around inside at once (or refuse the > commit if the change as a whole breaks the graph). I like this idea of the staging area. If a ticket or BZ happens, and someone remembers, please cc me :) Cheers! > > HTH, > Simo. > signature.asc Description: This is a digitally signed message part ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/06/2014 06:12 PM, Dmitri Pal wrote: On 06/06/2014 09:03 AM, Simo Sorce wrote: On Fri, 2014-06-06 at 06:58 -0400, James wrote: On Mon, Jun 2, 2014 at 4:46 AM, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree One comment to add: It would be particularly useful if the interface used to set the topology is something sane that a single host can use to set its peers. Eg: host1$ ipa-topology-set-peers host2 host2$ ipa-topology-set-peers host3 host3$ ipa-topology-set-peers host4 host4$ ipa-topology-set-peers host1 # redundant, but okay... This way config management could define topologies in terms of algorithms. Eg: Ring topology: Order hostnames alphabetically. Peer with host name index of self + 1 (Example shown above) Star topology: Peer with every other host in list Pick two topology: Peer with each subsequent hosts in ordered list... And so on... If running the command changes the topology again, then that's great because changing the algorithm would re-arrange the graph :) Hope this made sense. Oh it does! But I am dreaming even bigger, my aim is to end up (in the long run) with something like: ipa topology change --stellar host1 host2 host3 host4 host5 causes topology for those 5 servers only (ignores any other connection) to become: host2 | host3 - host1 - host5 | host4 So any agreement between host1 and 2,3,4,5 get wiped and replaced with the stellar topology. (agreements between 1,2,3,4,5 and 6,7,8,9 are not affected in any way and stay). And later on you'd be able to say ipa topology change --circular host1 host2 host3 host4 and you get: host1 - host2 | | host4 - host3 Note that if you previously did the stellar thing and you only have these 5 servers the actual complete topology ends up being like this: host5 | host1 - host2 | | host4 - host3 As the agreement between host1 and host5 is not touched by the second command. But this is in the far future IMO, we'll probably start by only implementing: ipa topology set-peering host1 host2 and ipa topology break-peering host3 host4 Ie creating and breaking segments in the topology tree only between pairs and storing/deleting the segment object in the shared tree. The reason is that the topology plugin will allow or disallow changes based on whether you are breaking the graph, and it is much simpler to do that if you change one object at a time. To do the nifty stuff above we'd need some staging area where we describe all the changes and then have a "commit" type change that would cause the topology plugin to do all the calculations and move stuff around inside at once (or refuse the commit if the change as a whole breaks the graph). HTH, Simo. I was thinking about the commit and staging too. I think this approach will be beneficial for the "rebuild from existing non replicated agreements" use case too. The logic for the use case that I described in other branch of this discussion will be: - Collect all info by contacting all the servers. - Save in staging - Analyze that topology makes sense and graph is not broken (error out if it is) - Start transaction - Put the data into the replicated tree - Stop transaction - Start replicating ok, so this will be handled outside of the topology plugin, a future step could be to define topology types and connectivity rules and have the topology plugin to verify this ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/06/2014 09:03 AM, Simo Sorce wrote: On Fri, 2014-06-06 at 06:58 -0400, James wrote: On Mon, Jun 2, 2014 at 4:46 AM, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree One comment to add: It would be particularly useful if the interface used to set the topology is something sane that a single host can use to set its peers. Eg: host1$ ipa-topology-set-peers host2 host2$ ipa-topology-set-peers host3 host3$ ipa-topology-set-peers host4 host4$ ipa-topology-set-peers host1 # redundant, but okay... This way config management could define topologies in terms of algorithms. Eg: Ring topology: Order hostnames alphabetically. Peer with host name index of self + 1 (Example shown above) Star topology: Peer with every other host in list Pick two topology: Peer with each subsequent hosts in ordered list... And so on... If running the command changes the topology again, then that's great because changing the algorithm would re-arrange the graph :) Hope this made sense. Oh it does! But I am dreaming even bigger, my aim is to end up (in the long run) with something like: ipa topology change --stellar host1 host2 host3 host4 host5 causes topology for those 5 servers only (ignores any other connection) to become: host2 | host3 - host1 - host5 | host4 So any agreement between host1 and 2,3,4,5 get wiped and replaced with the stellar topology. (agreements between 1,2,3,4,5 and 6,7,8,9 are not affected in any way and stay). And later on you'd be able to say ipa topology change --circular host1 host2 host3 host4 and you get: host1 - host2 | | host4 - host3 Note that if you previously did the stellar thing and you only have these 5 servers the actual complete topology ends up being like this: host5 | host1 - host2 | | host4 - host3 As the agreement between host1 and host5 is not touched by the second command. But this is in the far future IMO, we'll probably start by only implementing: ipa topology set-peering host1 host2 and ipa topology break-peering host3 host4 Ie creating and breaking segments in the topology tree only between pairs and storing/deleting the segment object in the shared tree. The reason is that the topology plugin will allow or disallow changes based on whether you are breaking the graph, and it is much simpler to do that if you change one object at a time. To do the nifty stuff above we'd need some staging area where we describe all the changes and then have a "commit" type change that would cause the topology plugin to do all the calculations and move stuff around inside at once (or refuse the commit if the change as a whole breaks the graph). HTH, Simo. I was thinking about the commit and staging too. I think this approach will be beneficial for the "rebuild from existing non replicated agreements" use case too. The logic for the use case that I described in other branch of this discussion will be: - Collect all info by contacting all the servers. - Save in staging - Analyze that topology makes sense and graph is not broken (error out if it is) - Start transaction - Put the data into the replicated tree - Stop transaction - Start replicating -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Fri, 2014-06-06 at 06:58 -0400, James wrote: > On Mon, Jun 2, 2014 at 4:46 AM, Ludwig Krispenz wrote: > > Ticket 4302 is a request for an enhancement: Move replication topology to > > the shared tree > > > One comment to add: > > It would be particularly useful if the interface used to set the > topology is something sane that a single host can use to set its > peers. Eg: > > host1$ ipa-topology-set-peers host2 > host2$ ipa-topology-set-peers host3 > host3$ ipa-topology-set-peers host4 > host4$ ipa-topology-set-peers host1 # redundant, but okay... > > This way config management could define topologies in terms of algorithms. Eg: > > Ring topology: > Order hostnames alphabetically. > Peer with host name index of self + 1 > (Example shown above) > > Star topology: > Peer with every other host in list > > Pick two topology: > Peer with each subsequent hosts in ordered list... > > And so on... > If running the command changes the topology again, then that's great > because changing the algorithm would re-arrange the graph :) > > Hope this made sense. Oh it does! But I am dreaming even bigger, my aim is to end up (in the long run) with something like: ipa topology change --stellar host1 host2 host3 host4 host5 causes topology for those 5 servers only (ignores any other connection) to become: host2 | host3 - host1 - host5 | host4 So any agreement between host1 and 2,3,4,5 get wiped and replaced with the stellar topology. (agreements between 1,2,3,4,5 and 6,7,8,9 are not affected in any way and stay). And later on you'd be able to say ipa topology change --circular host1 host2 host3 host4 and you get: host1 - host2 | | host4 - host3 Note that if you previously did the stellar thing and you only have these 5 servers the actual complete topology ends up being like this: host5 | host1 - host2 | | host4 - host3 As the agreement between host1 and host5 is not touched by the second command. But this is in the far future IMO, we'll probably start by only implementing: ipa topology set-peering host1 host2 and ipa topology break-peering host3 host4 Ie creating and breaking segments in the topology tree only between pairs and storing/deleting the segment object in the shared tree. The reason is that the topology plugin will allow or disallow changes based on whether you are breaking the graph, and it is much simpler to do that if you change one object at a time. To do the nifty stuff above we'd need some staging area where we describe all the changes and then have a "commit" type change that would cause the topology plugin to do all the calculations and move stuff around inside at once (or refuse the commit if the change as a whole breaks the graph). HTH, Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Mon, Jun 2, 2014 at 4:46 AM, Ludwig Krispenz wrote: > Ticket 4302 is a request for an enhancement: Move replication topology to > the shared tree One comment to add: It would be particularly useful if the interface used to set the topology is something sane that a single host can use to set its peers. Eg: host1$ ipa-topology-set-peers host2 host2$ ipa-topology-set-peers host3 host3$ ipa-topology-set-peers host4 host4$ ipa-topology-set-peers host1 # redundant, but okay... This way config management could define topologies in terms of algorithms. Eg: Ring topology: Order hostnames alphabetically. Peer with host name index of self + 1 (Example shown above) Star topology: Peer with every other host in list Pick two topology: Peer with each subsequent hosts in ordered list... And so on... If running the command changes the topology again, then that's great because changing the algorithm would re-arrange the graph :) Hope this made sense. Cheers, James ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
we need to be careful on the process, I have an idea how it could work, but need to think a bit more about it I am all ears. Simo. We already have several situations (CRL, DNSSEC, cert rotation) where a single server has to do the job first and all the rest should rely on that. We can simply our life by making the initialization a special admin initialized operation for the situations when we upgrade from earlier versions. So general topology plugin does not initialize anything automatically. If we build a new deployment the ipa replica management tools will drive the modifications as servers are added. If it is an upgrade admin might go to IPA configuration and ray build/rebuild topology. This will drop all segment information if there is any and using master list from cn=masters connect to each replica, query its replication agreements and create data for the replicated tree. If some replica is not on line the operation will report that replica can be connected and that topology is not complete. I think this is acceptable for topology plugin to focus only on keeping data in sync when replica management tools are invoked and mot worry about initialization after migration. I finally agree, especially after having done some experiments with a rapid prototype. If the topology plugin is started before the replication plugin and does changes to the shared tree, these are not replicated. If the topology plugin is started after the replication plugin I get a failure in replication that no csn is assigned, and the startup of the topology plugin fails. These might be bugs to be resolved, but I think I will follow Dmitri's scenario for now - improvements can come later ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/05/2014 01:12 PM, Simo Sorce wrote: On Thu, 2014-06-05 at 16:46 +0200, Ludwig Krispenz wrote: On 06/05/2014 03:14 PM, Ludwig Krispenz wrote: On 06/05/2014 02:45 PM, Simo Sorce wrote: On Thu, 2014-06-05 at 11:27 +0200, Ludwig Krispenz wrote: On 06/04/2014 06:04 PM, thierry bordaz wrote: But this requires that the node database is already initialized (have the same replicageneration than the others nodes). If it is not initialized, it could be done by any masters. But if it is automatic, there is a risk that several masters will want to initialize it. I would not give the plugin the power to reinitialize a database on another server and I also would not put the responsibility to do it to the plugin. Initializing a server is an administrative task done at specific steps during deployment or in error conditions and most time has to be scheduled and often on-line initialization is not the preferred method. Agree. The plugin could still be used to trigger online initializations if the nsds5replicarefresh attribute is part of the information in the shared tree, it can be modified, the plugin on the targeted server will detect the change, update the replication agreement object and start the initialization (but this should probably still be allowed to be done directly). Nope, leave re-initialization to the plumbing. The topology plugin deals only with topology, it should not be involved with re-initializations, save for not going crazy and trying to remove agreements "during" a re-initialization. The question for me was more how the configuration information in the shared tree is initialized (not the full shared tree). We do agree that the info in the shared tree should be authoritative. Yep. To synchronize the info in the shared tree and cn=config I see two modes: "passive" mode: the sync is only from the shared tree to cn=config, it is the responsibility of an admin/tool to initialize the config in the shared tree This is my preferred, although leaves the problem open for migration, we need a way to properly prime the topology so that it doesn't wipe out current agreements before they are imported. "supporting" mode: if the plugin starts, it checks if the info in the shared tree exists, if not it initializes the topology info in the shared tree and then only reacts to changes in the shared tree. I would like some more details about what you have in mind here, depending on the fine points I might agree this is desirable to solve the migration problem. I will write it down for a few different scenarios. looks like this could get ugly, if the topology info initialization would happen on several masters at the same time, they would create the same entries (at least the config root container) and we could get replication conflicts :-( This is exactly why I said I do not want the plugin to create the topology tree itself :-) It should only import agreements that are not yet there. There is still potential conflict, the topology plugin will have to handle them, by intercepting replication writes to the topology tree and smartly merging/fencing IMO. we need to be careful on the process, I have an idea how it could work, but need to think a bit more about it I am all ears. Simo. We already have several situations (CRL, DNSSEC, cert rotation) where a single server has to do the job first and all the rest should rely on that. We can simply our life by making the initialization a special admin initialized operation for the situations when we upgrade from earlier versions. So general topology plugin does not initialize anything automatically. If we build a new deployment the ipa replica management tools will drive the modifications as servers are added. If it is an upgrade admin might go to IPA configuration and ray build/rebuild topology. This will drop all segment information if there is any and using master list from cn=masters connect to each replica, query its replication agreements and create data for the replicated tree. If some replica is not on line the operation will report that replica can be connected and that topology is not complete. I think this is acceptable for topology plugin to focus only on keeping data in sync when replica management tools are invoked and mot worry about initialization after migration. -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Thu, 2014-06-05 at 16:46 +0200, Ludwig Krispenz wrote: > On 06/05/2014 03:14 PM, Ludwig Krispenz wrote: > > > > On 06/05/2014 02:45 PM, Simo Sorce wrote: > >> On Thu, 2014-06-05 at 11:27 +0200, Ludwig Krispenz wrote: > >>> On 06/04/2014 06:04 PM, thierry bordaz wrote: > But this requires that the node database is already initialized (have > the same replicageneration than the others nodes). > If it is not initialized, it could be done by any masters. But if it > is automatic, there is a risk that several masters will want to > initialize it. > >>> I would not give the plugin the power to reinitialize a database on > >>> another server and I also would not put the responsibility to do it to > >>> the plugin. Initializing a server is an administrative task done at > >>> specific steps during deployment or in error conditions and most time > >>> has to be scheduled and often on-line initialization is not the > >>> preferred method. > >> Agree. > >> > >>> The plugin could still be used to trigger online initializations if the > >>> nsds5replicarefresh attribute is part of the information in the shared > >>> tree, it can be modified, the plugin on the targeted server will detect > >>> the change, update the replication agreement object and start the > >>> initialization (but this should probably still be allowed to be done > >>> directly). > >> Nope, leave re-initialization to the plumbing. The topology plugin deals > >> only with topology, it should not be involved with re-initializations, > >> save for not going crazy and trying to remove agreements "during" a > >> re-initialization. > >> > >>> The question for me was more how the configuration information in the > >>> shared tree is initialized (not the full shared tree). > >>> We do agree that the info in the shared tree should be authoritative. > >> Yep. > >> > >>> To > >>> synchronize the info in the shared tree and cn=config I see two modes: > >>> "passive" mode: the sync is only from the shared tree to cn=config, it > >>> is the responsibility of an admin/tool to initialize the config in the > >>> shared tree > >> This is my preferred, although leaves the problem open for migration, we > >> need a way to properly prime the topology so that it doesn't wipe out > >> current agreements before they are imported. > >> > >>> "supporting" mode: if the plugin starts, it checks if the info in the > >>> shared tree exists, if not it initializes the topology info in the > >>> shared tree and then only reacts to changes in the shared tree. > >> I would like some more details about what you have in mind here, > >> depending on the fine points I might agree this is desirable to solve > >> the migration problem. > > I will write it down for a few different scenarios. > looks like this could get ugly, if the topology info initialization > would happen on several masters at the same time, they would create the > same entries (at least the config root container) and we could get > replication conflicts :-( This is exactly why I said I do not want the plugin to create the topology tree itself :-) It should only import agreements that are not yet there. There is still potential conflict, the topology plugin will have to handle them, by intercepting replication writes to the topology tree and smartly merging/fencing IMO. > we need to be careful on the process, I have an idea how it could work, > but need to think a bit more about it I am all ears. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/05/2014 03:14 PM, Ludwig Krispenz wrote: On 06/05/2014 02:45 PM, Simo Sorce wrote: On Thu, 2014-06-05 at 11:27 +0200, Ludwig Krispenz wrote: On 06/04/2014 06:04 PM, thierry bordaz wrote: But this requires that the node database is already initialized (have the same replicageneration than the others nodes). If it is not initialized, it could be done by any masters. But if it is automatic, there is a risk that several masters will want to initialize it. I would not give the plugin the power to reinitialize a database on another server and I also would not put the responsibility to do it to the plugin. Initializing a server is an administrative task done at specific steps during deployment or in error conditions and most time has to be scheduled and often on-line initialization is not the preferred method. Agree. The plugin could still be used to trigger online initializations if the nsds5replicarefresh attribute is part of the information in the shared tree, it can be modified, the plugin on the targeted server will detect the change, update the replication agreement object and start the initialization (but this should probably still be allowed to be done directly). Nope, leave re-initialization to the plumbing. The topology plugin deals only with topology, it should not be involved with re-initializations, save for not going crazy and trying to remove agreements "during" a re-initialization. The question for me was more how the configuration information in the shared tree is initialized (not the full shared tree). We do agree that the info in the shared tree should be authoritative. Yep. To synchronize the info in the shared tree and cn=config I see two modes: "passive" mode: the sync is only from the shared tree to cn=config, it is the responsibility of an admin/tool to initialize the config in the shared tree This is my preferred, although leaves the problem open for migration, we need a way to properly prime the topology so that it doesn't wipe out current agreements before they are imported. "supporting" mode: if the plugin starts, it checks if the info in the shared tree exists, if not it initializes the topology info in the shared tree and then only reacts to changes in the shared tree. I would like some more details about what you have in mind here, depending on the fine points I might agree this is desirable to solve the migration problem. I will write it down for a few different scenarios. looks like this could get ugly, if the topology info initialization would happen on several masters at the same time, they would create the same entries (at least the config root container) and we could get replication conflicts :-( we need to be careful on the process, I have an idea how it could work, but need to think a bit more about it I prefer the "supporting" mode (if we find a better name), as long as no admin interferes the info in the shared tree and cn=config will be identical as they are set in a normal deployment, so no harm is done, but they are replicated and the full information is available on every server. Full topology information you mean here, right ? topology information (nodes and edges) and properties of the connections, eg replicated or stripped attributes Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/05/2014 02:45 PM, Simo Sorce wrote: On Thu, 2014-06-05 at 11:27 +0200, Ludwig Krispenz wrote: On 06/04/2014 06:04 PM, thierry bordaz wrote: But this requires that the node database is already initialized (have the same replicageneration than the others nodes). If it is not initialized, it could be done by any masters. But if it is automatic, there is a risk that several masters will want to initialize it. I would not give the plugin the power to reinitialize a database on another server and I also would not put the responsibility to do it to the plugin. Initializing a server is an administrative task done at specific steps during deployment or in error conditions and most time has to be scheduled and often on-line initialization is not the preferred method. Agree. The plugin could still be used to trigger online initializations if the nsds5replicarefresh attribute is part of the information in the shared tree, it can be modified, the plugin on the targeted server will detect the change, update the replication agreement object and start the initialization (but this should probably still be allowed to be done directly). Nope, leave re-initialization to the plumbing. The topology plugin deals only with topology, it should not be involved with re-initializations, save for not going crazy and trying to remove agreements "during" a re-initialization. The question for me was more how the configuration information in the shared tree is initialized (not the full shared tree). We do agree that the info in the shared tree should be authoritative. Yep. To synchronize the info in the shared tree and cn=config I see two modes: "passive" mode: the sync is only from the shared tree to cn=config, it is the responsibility of an admin/tool to initialize the config in the shared tree This is my preferred, although leaves the problem open for migration, we need a way to properly prime the topology so that it doesn't wipe out current agreements before they are imported. "supporting" mode: if the plugin starts, it checks if the info in the shared tree exists, if not it initializes the topology info in the shared tree and then only reacts to changes in the shared tree. I would like some more details about what you have in mind here, depending on the fine points I might agree this is desirable to solve the migration problem. I will write it down for a few different scenarios. I prefer the "supporting" mode (if we find a better name), as long as no admin interferes the info in the shared tree and cn=config will be identical as they are set in a normal deployment, so no harm is done, but they are replicated and the full information is available on every server. Full topology information you mean here, right ? topology information (nodes and edges) and properties of the connections, eg replicated or stripped attributes Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Thu, 2014-06-05 at 11:27 +0200, Ludwig Krispenz wrote: > On 06/04/2014 06:04 PM, thierry bordaz wrote: > > But this requires that the node database is already initialized (have > > the same replicageneration than the others nodes). > > If it is not initialized, it could be done by any masters. But if it > > is automatic, there is a risk that several masters will want to > > initialize it. > I would not give the plugin the power to reinitialize a database on > another server and I also would not put the responsibility to do it to > the plugin. Initializing a server is an administrative task done at > specific steps during deployment or in error conditions and most time > has to be scheduled and often on-line initialization is not the > preferred method. Agree. > The plugin could still be used to trigger online initializations if the > nsds5replicarefresh attribute is part of the information in the shared > tree, it can be modified, the plugin on the targeted server will detect > the change, update the replication agreement object and start the > initialization (but this should probably still be allowed to be done > directly). Nope, leave re-initialization to the plumbing. The topology plugin deals only with topology, it should not be involved with re-initializations, save for not going crazy and trying to remove agreements "during" a re-initialization. > The question for me was more how the configuration information in the > shared tree is initialized (not the full shared tree). > We do agree that the info in the shared tree should be authoritative. Yep. > To > synchronize the info in the shared tree and cn=config I see two modes: > "passive" mode: the sync is only from the shared tree to cn=config, it > is the responsibility of an admin/tool to initialize the config in the > shared tree This is my preferred, although leaves the problem open for migration, we need a way to properly prime the topology so that it doesn't wipe out current agreements before they are imported. > "supporting" mode: if the plugin starts, it checks if the info in the > shared tree exists, if not it initializes the topology info in the > shared tree and then only reacts to changes in the shared tree. I would like some more details about what you have in mind here, depending on the fine points I might agree this is desirable to solve the migration problem. > I prefer the "supporting" mode (if we find a better name), as long as no > admin interferes the info in the shared tree and cn=config will be > identical as they are set in a normal deployment, so no harm is done, > but they are replicated and the full information is available on every > server. Full topology information you mean here, right ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/04/2014 06:04 PM, thierry bordaz wrote: On 06/04/2014 05:41 PM, Simo Sorce wrote: On Wed, 2014-06-04 at 13:46 +0200, Ludwig Krispenz wrote: On 06/04/2014 10:43 AM, thierry bordaz wrote: So my proposal would contain the following components 1] Store replication configuration in the shared tree in a combination of server and connection view (think we need both) and map replication configuration to these entries. I would prefer a direct mapping (with a subset of the cn=config attributes and required additions) About adding a new server in the replication topology. If it was initialized, it may register itself into the shared tree and then join the topology. If it was not initialized, it can be initialized online by one of the master. Will it be triggered with an update to the shared tree ? I think this has to be decided, I proposed some kind of bootstrapping: If the topology plugin is enabled and started it would check if there is already info on the connections for this server and if not create/update the entry in the shared tree, this could also happen if a new server is added. You mean updating the shared tree from information about replication agreements found in cn=config ? I can see how this would be useful in migration from previous server versions, but I fear would cause issues if you remove a connection when one of the 2 servers is down. When you bring the server up it would try to re-create a connection that was deleted by the admin. It's a catch-22 which is why I want the shared tree to be authoritative but not the cn=config tree. But Simo wanted to have the info in the shared tree indepndent of what was already configured. It's not much about being independent, it's about what is authoritative and what is not. One problem with the automatic approach is what should be done if the config differs on the already deployed servers That's just one of many problems. I think cn=config entries should be read an injected in the shared topology only once at upgrade time, but not anymore after. Maybe this calls for adding nodes to the topology tree, if the topology plugin starts and the server is not in the topology tree, then it sources the local configuration, then adds itself and the connections to the topology tree. If the node is already in the topology tree this step is always skipped. I like the idea that the node can add itself and the connection to the topology tree. But this requires that the node database is already initialized (have the same replicageneration than the others nodes). If it is not initialized, it could be done by any masters. But if it is automatic, there is a risk that several masters will want to initialize it. I would not give the plugin the power to reinitialize a database on another server and I also would not put the responsibility to do it to the plugin. Initializing a server is an administrative task done at specific steps during deployment or in error conditions and most time has to be scheduled and often on-line initialization is not the preferred method. The plugin could still be used to trigger online initializations if the nsds5replicarefresh attribute is part of the information in the shared tree, it can be modified, the plugin on the targeted server will detect the change, update the replication agreement object and start the initialization (but this should probably still be allowed to be done directly). The question for me was more how the configuration information in the shared tree is initialized (not the full shared tree). We do agree that the info in the shared tree should be authoritative. To synchronize the info in the shared tree and cn=config I see two modes: "passive" mode: the sync is only from the shared tree to cn=config, it is the responsibility of an admin/tool to initialize the config in the shared tree "supporting" mode: if the plugin starts, it checks if the info in the shared tree exists, if not it initializes the topology info in the shared tree and then only reacts to changes in the shared tree. I prefer the "supporting" mode (if we find a better name), as long as no admin interferes the info in the shared tree and cn=config will be identical as they are set in a normal deployment, so no harm is done, but they are replicated and the full information is available on every server. In addition if the new node has a different replicageneration, how does the new node know that it should wait to be initialized rather than initialize the others. Or something similar (could be just a flag in cn=masters objects). makes sense ? Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Wed, 2014-06-04 at 18:04 +0200, thierry bordaz wrote: > On 06/04/2014 05:41 PM, Simo Sorce wrote: > > On Wed, 2014-06-04 at 13:46 +0200, Ludwig Krispenz wrote: > >> On 06/04/2014 10:43 AM, thierry bordaz wrote: > So my proposal would contain the following components > > 1] Store replication configuration in the shared tree in a > combination of server and connection view (think we need both) and > map replication configuration to these entries. I would prefer a > direct mapping (with a subset of the cn=config attributes and > required additions) > >>> About adding a new server in the replication topology. > >>> If it was initialized, it may register itself into the shared tree and > >>> then join the topology. If it was not initialized, it can be > >>> initialized online by one of the master. Will it be triggered with an > >>> update to the shared tree ? > >> I think this has to be decided, I proposed some kind of bootstrapping: > >> If the topology plugin is enabled and started it would check if there is > >> already info on the connections for this server and if not create/update > >> the entry in the shared tree, this could also happen if a new server is > >> added. > > You mean updating the shared tree from information about replication > > agreements found in cn=config ? > > I can see how this would be useful in migration from previous server > > versions, but I fear would cause issues if you remove a connection when > > one of the 2 servers is down. > > When you bring the server up it would try to re-create a connection that > > was deleted by the admin. It's a catch-22 which is why I want the shared > > tree to be authoritative but not the cn=config tree. > > > >> But Simo wanted to have the info in the shared tree indepndent of what > >> was already configured. > > It's not much about being independent, it's about what is authoritative > > and what is not. > > > >> One problem with the automatic approach is what should be done if the > >> config differs on the already deployed servers > > That's just one of many problems. > > I think cn=config entries should be read an injected in the shared > > topology only once at upgrade time, but not anymore after. > > Maybe this calls for adding nodes to the topology tree, if the topology > > plugin starts and the server is not in the topology tree, then it > > sources the local configuration, then adds itself and the connections to > > the topology tree. > > If the node is already in the topology tree this step is always skipped. > I like the idea that the node can add itself and the connection to the > topology tree. > But this requires that the node database is already initialized (have > the same replicageneration than the others nodes). > If it is not initialized, it could be done by any masters. But if it is > automatic, there is a risk that several masters will want to initialize it. > > In addition if the new node has a different replicageneration, how does > the new node know that it should wait to be initialized rather than > initialize the others. Yeah see my follow-up, I think a flag would be better, this way import is done only once and never more. If a node is already initialized then the topology plugin will always wipe away and reconfigure agreements in cn=config from the topology tree and never backwards. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/04/2014 05:41 PM, Simo Sorce wrote: On Wed, 2014-06-04 at 13:46 +0200, Ludwig Krispenz wrote: On 06/04/2014 10:43 AM, thierry bordaz wrote: So my proposal would contain the following components 1] Store replication configuration in the shared tree in a combination of server and connection view (think we need both) and map replication configuration to these entries. I would prefer a direct mapping (with a subset of the cn=config attributes and required additions) About adding a new server in the replication topology. If it was initialized, it may register itself into the shared tree and then join the topology. If it was not initialized, it can be initialized online by one of the master. Will it be triggered with an update to the shared tree ? I think this has to be decided, I proposed some kind of bootstrapping: If the topology plugin is enabled and started it would check if there is already info on the connections for this server and if not create/update the entry in the shared tree, this could also happen if a new server is added. You mean updating the shared tree from information about replication agreements found in cn=config ? I can see how this would be useful in migration from previous server versions, but I fear would cause issues if you remove a connection when one of the 2 servers is down. When you bring the server up it would try to re-create a connection that was deleted by the admin. It's a catch-22 which is why I want the shared tree to be authoritative but not the cn=config tree. But Simo wanted to have the info in the shared tree indepndent of what was already configured. It's not much about being independent, it's about what is authoritative and what is not. One problem with the automatic approach is what should be done if the config differs on the already deployed servers That's just one of many problems. I think cn=config entries should be read an injected in the shared topology only once at upgrade time, but not anymore after. Maybe this calls for adding nodes to the topology tree, if the topology plugin starts and the server is not in the topology tree, then it sources the local configuration, then adds itself and the connections to the topology tree. If the node is already in the topology tree this step is always skipped. I like the idea that the node can add itself and the connection to the topology tree. But this requires that the node database is already initialized (have the same replicageneration than the others nodes). If it is not initialized, it could be done by any masters. But if it is automatic, there is a risk that several masters will want to initialize it. In addition if the new node has a different replicageneration, how does the new node know that it should wait to be initialized rather than initialize the others. Or something similar (could be just a flag in cn=masters objects). makes sense ? Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Wed, 2014-06-04 at 11:41 -0400, Simo Sorce wrote: > On Wed, 2014-06-04 at 13:46 +0200, Ludwig Krispenz wrote: > > On 06/04/2014 10:43 AM, thierry bordaz wrote: > > > > > >> > > >> So my proposal would contain the following components > > >> > > >> 1] Store replication configuration in the shared tree in a > > >> combination of server and connection view (think we need both) and > > >> map replication configuration to these entries. I would prefer a > > >> direct mapping (with a subset of the cn=config attributes and > > >> required additions) > > > > > > About adding a new server in the replication topology. > > > If it was initialized, it may register itself into the shared tree and > > > then join the topology. If it was not initialized, it can be > > > initialized online by one of the master. Will it be triggered with an > > > update to the shared tree ? > > > > I think this has to be decided, I proposed some kind of bootstrapping: > > If the topology plugin is enabled and started it would check if there is > > already info on the connections for this server and if not create/update > > the entry in the shared tree, this could also happen if a new server is > > added. > > You mean updating the shared tree from information about replication > agreements found in cn=config ? > I can see how this would be useful in migration from previous server > versions, but I fear would cause issues if you remove a connection when > one of the 2 servers is down. > When you bring the server up it would try to re-create a connection that > was deleted by the admin. It's a catch-22 which is why I want the shared > tree to be authoritative but not the cn=config tree. > > > But Simo wanted to have the info in the shared tree indepndent of what > > was already configured. > > It's not much about being independent, it's about what is authoritative > and what is not. > > > One problem with the automatic approach is what should be done if the > > config differs on the already deployed servers > > That's just one of many problems. > I think cn=config entries should be read an injected in the shared > topology only once at upgrade time, but not anymore after. > Maybe this calls for adding nodes to the topology tree, if the topology > plugin starts and the server is not in the topology tree, then it > sources the local configuration, then adds itself and the connections to > the topology tree. > If the node is already in the topology tree this step is always skipped. > > Or something similar (could be just a flag in cn=masters objects). Replying to myself, a flag on the cn=masters servers is probably better, because otherwise connection objects would reference "missing" nodes (servers that have not already been upgraded) or they would cause the creation of missing node (for integrity) and that would cause those servers to not be able to import their agreements. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Wed, 2014-06-04 at 13:46 +0200, Ludwig Krispenz wrote: > On 06/04/2014 10:43 AM, thierry bordaz wrote: > > > >> > >> So my proposal would contain the following components > >> > >> 1] Store replication configuration in the shared tree in a > >> combination of server and connection view (think we need both) and > >> map replication configuration to these entries. I would prefer a > >> direct mapping (with a subset of the cn=config attributes and > >> required additions) > > > > About adding a new server in the replication topology. > > If it was initialized, it may register itself into the shared tree and > > then join the topology. If it was not initialized, it can be > > initialized online by one of the master. Will it be triggered with an > > update to the shared tree ? > > I think this has to be decided, I proposed some kind of bootstrapping: > If the topology plugin is enabled and started it would check if there is > already info on the connections for this server and if not create/update > the entry in the shared tree, this could also happen if a new server is > added. You mean updating the shared tree from information about replication agreements found in cn=config ? I can see how this would be useful in migration from previous server versions, but I fear would cause issues if you remove a connection when one of the 2 servers is down. When you bring the server up it would try to re-create a connection that was deleted by the admin. It's a catch-22 which is why I want the shared tree to be authoritative but not the cn=config tree. > But Simo wanted to have the info in the shared tree indepndent of what > was already configured. It's not much about being independent, it's about what is authoritative and what is not. > One problem with the automatic approach is what should be done if the > config differs on the already deployed servers That's just one of many problems. I think cn=config entries should be read an injected in the shared topology only once at upgrade time, but not anymore after. Maybe this calls for adding nodes to the topology tree, if the topology plugin starts and the server is not in the topology tree, then it sources the local configuration, then adds itself and the connections to the topology tree. If the node is already in the topology tree this step is always skipped. Or something similar (could be just a flag in cn=masters objects). makes sense ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Wed, 2014-06-04 at 13:50 +0200, Ludwig Krispenz wrote: > my question was more where the r/o replicas would be listed. In > cn=ro-replicas parallel to cn=matsters, > or should all servers be in cn=masters and have a replication role defined ? We do not know yet as we do not have read-only replicas at all. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/03/2014 05:59 PM, Simo Sorce wrote: On Tue, 2014-06-03 at 15:47 +0200, Ludwig Krispenz wrote: On 06/03/2014 02:53 PM, Simo Sorce wrote: On Tue, 2014-06-03 at 14:15 +0200, Ludwig Krispenz wrote: Hi Simo, just for clarification. The plan is to move the repl config into the shared tree for the main database and eventually for others like o=ipaca. Should the topology info live in cn=etc for all databases or each in the database it configures ? If the main database is always replicated I think all topology info should be concentrated in cn=etc. This is actually a good question, I think we want to keep it in the main database, but to be forward looking, we should set the basedn of the configuration (per topology) in the topology plugin configuration entry, this way if it turns out we should move it in their own dbs we can just do it. yes, this can be handle in the topology plugin configuration, we need to define which databases should be controlled and where the root for the replicated topology information is. I assume we want only one plugin instance. The other issue is the list of masters though. I do not want to move them in the topology tree (breaks compatibility) nor duplicate those entries if possible, but we may need to do it or also configure where the "nodes" are stored. Agreed. What about non-masters, in the ticket there was also discussion about read only replicas - to be delayed ? TBD, from the pov of the plugin we care only about Nodes and links, but yeah dealing with R/O replicas will mean the plugin should understand such servers need a segment that sends data to them but do not need a segment that carries data out. my question was more where the r/o replicas would be listed. In cn=ro-replicas parallel to cn=matsters, or should all servers be in cn=masters and have a replication role defined ? Working on the topology consistency checker will be fun :-) Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/04/2014 10:43 AM, thierry bordaz wrote: So my proposal would contain the following components 1] Store replication configuration in the shared tree in a combination of server and connection view (think we need both) and map replication configuration to these entries. I would prefer a direct mapping (with a subset of the cn=config attributes and required additions) About adding a new server in the replication topology. If it was initialized, it may register itself into the shared tree and then join the topology. If it was not initialized, it can be initialized online by one of the master. Will it be triggered with an update to the shared tree ? I think this has to be decided, I proposed some kind of bootstrapping: If the topology plugin is enabled and started it would check if there is already info on the connections for this server and if not create/update the entry in the shared tree, this could also happen if a new server is added. But Simo wanted to have the info in the shared tree indepndent of what was already configured. One problem with the automatic approach is what should be done if the config differs on the already deployed servers ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/02/2014 10:46 AM, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree There has been some discussion in comments in the ticket, but I'd like to open the discussion to a wider audience to get an agreement on what should be implemented, before writing a design spec. The implementation requires a new IPA plugin for 389 DS and eventually an enhancement of the 389 replication plugin (which depends on some decisions below). In the following I will use the terms “topology plugin” for the new plugin and “replication plugin” for the existing 389 multimaster replication plugin. Lets start with the requirements: What should be achieved by this RFE ? In my opinion there are three different levels of features to implement with this request - providing all replication configuration information consistent across all deployed servers on all servers, eg to easily visualize the replication topology. - Allowing to do sanity checks on replication configuration, denying modifications which would break replication topology or issue warnings. - Use the information in the shared tree to trigger changes to the replication configuration in the corresponding servers, this means to allow to completely control replication configuration with modifications of entries in the shared tree The main questions are 1] which information is needed in the shared tree (eg what params in the repl config should be modifiable) 2] how is the information organized and stored (layout of the repl config information shared tree) 3] how is the interaction of the info in the shared tree and configuration in cn=config and the interaction between the topology plugin and the replication plugin ad 1] to verify the topology, eg connectivity info about all existing replication agreements is needed, the replication agreements only contain info about the target, and the parameters for connection to the target, but not about the origin. If the data have to evaluated on any server, information about the origin has to be added, eg replicaID, serverID,... In addition, if agreement config has to be changed based on the shared tree all required parameters need to be present, eg replicatedAttributeList, strippedAttrs, replicationEnabled, . Replication agreements only provide information on connections where replication is configured, if connectivity is to be checked independent info about all deployed serevers/replicas is needed. If topology should be validated, do we need params definig requirements, eg each replica to be connected to 1,2,3,... others, type of topology (ring, mesh, star,.) ? ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. In my opinion a 1:1 relation is straight forward, easy to handle and easy to extend (not the full data of a repl agreement need to be present, other attributes are possible). The downside may be a larger number of entries, but this is no problem for the directory server and replication and the utilities eg to visualize a topology will handle this. If the number of entries should be reduced information on multiple replication agreements would have to be stored in one entry, and the problem arises ho to group data belonging to one agreement. LDAP does not provide a simple way to group attribute values in one entry, so all the info related to one agreement (origin, target, replicated attrs and other repl configuration info) could be stored in a single attribute, which will make the attribute as nicely readable and managable as acis. If topology verification and connectivity check is an integral part of the feature, I think a connection oriented view is not sufficient, it might be incomplete, so a server view is required, the server entry would then have the connection information as subentries or as attributes. Ad 3] The replication configuration is stored under cn=config and can be modified either by ldap operations or by editing the dse.ldif. With the topology plugin another source of configuration changes comes into play. The first question is: which information has precendence ? I think if there is info in the shared tree it should be used, and the information in cn=config should be updated. This also means that the topology plugin needs to intercept all mods to the entries in cn=config and have them ignored and handle all updates to the shared tree and trigger changes to the cn=config entries, which then would trigger rebuilds of the in memory replication objects. Next question: How to handle changes directly done in the dse.ldif, if everything should be done by the topology plugin it would have to verify and compare the info in cn=con
Re: [Freeipa-devel] Move replication topology to the shared tree
On Tue, 2014-06-03 at 15:47 +0200, Ludwig Krispenz wrote: > On 06/03/2014 02:53 PM, Simo Sorce wrote: > > On Tue, 2014-06-03 at 14:15 +0200, Ludwig Krispenz wrote: > >> Hi Simo, > >> > >> just for clarification. The plan is to move the repl config into the > >> shared tree for the main database and eventually for others like > >> o=ipaca. Should the topology info live in cn=etc for all databases or > >> each in the database it configures ? > >> If the main database is always replicated I think all topology info > >> should be concentrated in cn=etc. > > This is actually a good question, I think we want to keep it in the main > > database, but to be forward looking, we should set the basedn of the > > configuration (per topology) in the topology plugin configuration entry, > > this way if it turns out we should move it in their own dbs we can just > > do it. > yes, this can be handle in the topology plugin configuration, we need to > define which databases should be controlled and where the root for the > replicated topology information is. > I assume we want only one plugin instance. > > > > The other issue is the list of masters though. I do not want to move > > them in the topology tree (breaks compatibility) nor duplicate those > > entries if possible, but we may need to do it or also configure where > > the "nodes" are stored. > Agreed. What about non-masters, in the ticket there was also discussion > about read only replicas - to be delayed ? TBD, from the pov of the plugin we care only about Nodes and links, but yeah dealing with R/O replicas will mean the plugin should understand such servers need a segment that sends data to them but do not need a segment that carries data out. Working on the topology consistency checker will be fun :-) Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/03/2014 02:53 PM, Simo Sorce wrote: On Tue, 2014-06-03 at 14:15 +0200, Ludwig Krispenz wrote: Hi Simo, just for clarification. The plan is to move the repl config into the shared tree for the main database and eventually for others like o=ipaca. Should the topology info live in cn=etc for all databases or each in the database it configures ? If the main database is always replicated I think all topology info should be concentrated in cn=etc. This is actually a good question, I think we want to keep it in the main database, but to be forward looking, we should set the basedn of the configuration (per topology) in the topology plugin configuration entry, this way if it turns out we should move it in their own dbs we can just do it. yes, this can be handle in the topology plugin configuration, we need to define which databases should be controlled and where the root for the replicated topology information is. I assume we want only one plugin instance. The other issue is the list of masters though. I do not want to move them in the topology tree (breaks compatibility) nor duplicate those entries if possible, but we may need to do it or also configure where the "nodes" are stored. Agreed. What about non-masters, in the ticket there was also discussion about read only replicas - to be delayed ? Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On Tue, 2014-06-03 at 14:15 +0200, Ludwig Krispenz wrote: > Hi Simo, > > just for clarification. The plan is to move the repl config into the > shared tree for the main database and eventually for others like > o=ipaca. Should the topology info live in cn=etc for all databases or > each in the database it configures ? > If the main database is always replicated I think all topology info > should be concentrated in cn=etc. This is actually a good question, I think we want to keep it in the main database, but to be forward looking, we should set the basedn of the configuration (per topology) in the topology plugin configuration entry, this way if it turns out we should move it in their own dbs we can just do it. The other issue is the list of masters though. I do not want to move them in the topology tree (breaks compatibility) nor duplicate those entries if possible, but we may need to do it or also configure where the "nodes" are stored. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
Hi Simo, just for clarification. The plan is to move the repl config into the shared tree for the main database and eventually for others like o=ipaca. Should the topology info live in cn=etc for all databases or each in the database it configures ? If the main database is always replicated I think all topology info should be concentrated in cn=etc. Ludwig On 06/02/2014 04:38 PM, Simo Sorce wrote: On Mon, 2014-06-02 at 10:08 -0400, Rob Crittenden wrote: Simo Sorce wrote: However we may want to be able to mark a topology for 'multiple' sets. For example we may want to have by default the same topology both for the main database and for the CA database. I think we should store them separately and making them the "same" would be applied by a tool, but the data would just reflect the connections. I was thinking the object DN would contain the LDAP database name (or some rough equivalent), so we would store the IPA connections separate from the CA connections. Ok, we can debate about this, maybe we simply have a flag in the framework that 'links' two topologies and simply replicates any change from one to the other. The only reason I had to 'mark' stuff in a single topology in order to share it is that this way any change is atomic and the 2 topologies cannot diverge, as the objects are the same, if we think the chance of divergence is low or that it is not important because the topology plugin will always prevent disconnected states anyway then we may avoid it, and let the framework try to keep topologies in sync and just loudly warn if they somehow get out of sync (which will happen briefly every time replication of the topology objects happens :). ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. My answer is no, we need only one object per connection, but config entries are per direction (and different ones on different servers). We also need to store the type, MMR, read-only, etc, for future-proofing. Store where ? One entry per connection would mirror what we have now in the mapping tree (which is generally ok). I wonder if this would be limiting with other agreement types depending on the schema we use. My idea is that on the connection object you have a set of attributes that tells you how replication happen. So normally you'll have: dn: uuid? objectclass: ipaReplicationTopologySegment left: master-X right: master-Y direction: both || left-right || right-left (|| none ?) If we have other special types we change direction accordingly or add another attribute. We already have the list of servers, so we need to add only the list of connections in the topology view. We may need to amend the servers objects to add additional data in some cases. For example indicate whether it is fully installed or not (on creation the topology plugin would complain the server is disconnected until we create the first segment, but that may actually be a good thing :-) Not sure I grok the fully installed part. A server isn't added as a master until it is actually installed, so a prepared master shouldn't show here. Uhmm you may be right, if we can make this a non-problem, all the better. Next question: How to handle changes directly done in the dse.ldif, if everything should be done by the topology plugin it would have to verify and compare the info in cn=config and in the shared tree at every startup of the directory server, which might be complicated by the fact that the replication plugin might already be started and repl agreemnts are active before the topology plugin is started and could do its work. (plugin starting order and dependencies need to be checked). Why do we care which one starts first ? We can simply change replication agreements at any time, so the fact the replication topology (and therefore agreements) can change after startup should not be an issue. Someone could delete an agreement, or worse, add one we don't know about. Does that matter? Someone can do this at any time after startup, so we already need to handle this, why should it be a problem ? However I agree we want to avoid churn, so to answer to Ludwig as well I guess we just want to make sure the topology plugin always starts before the replication plugin and amends replication agreements accordingly. What happens to values in the mapping tree that aren't represented in our own topology view? I think we should ignore them if they reference a machine that is not a recognized master, I guess the main issue here is a case when a master got deleted and somehow the cn=config entry was not and we end up with an orphan agreement that the topology plugin initially created but does not recognize as its own. I see 2 options here: 1) We ignore it, and let the admin deal with the issue 2) We mark a
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/02/2014 08:38 AM, Simo Sorce wrote: On Mon, 2014-06-02 at 10:08 -0400, Rob Crittenden wrote: Simo Sorce wrote: However we may want to be able to mark a topology for 'multiple' sets. For example we may want to have by default the same topology both for the main database and for the CA database. I think we should store them separately and making them the "same" would be applied by a tool, but the data would just reflect the connections. I was thinking the object DN would contain the LDAP database name (or some rough equivalent), so we would store the IPA connections separate from the CA connections. Ok, we can debate about this, maybe we simply have a flag in the framework that 'links' two topologies and simply replicates any change from one to the other. The only reason I had to 'mark' stuff in a single topology in order to share it is that this way any change is atomic and the 2 topologies cannot diverge, as the objects are the same, if we think the chance of divergence is low or that it is not important because the topology plugin will always prevent disconnected states anyway then we may avoid it, and let the framework try to keep topologies in sync and just loudly warn if they somehow get out of sync (which will happen briefly every time replication of the topology objects happens :). ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. My answer is no, we need only one object per connection, but config entries are per direction (and different ones on different servers). We also need to store the type, MMR, read-only, etc, for future-proofing. Store where ? One entry per connection would mirror what we have now in the mapping tree (which is generally ok). I wonder if this would be limiting with other agreement types depending on the schema we use. My idea is that on the connection object you have a set of attributes that tells you how replication happen. So normally you'll have: dn: uuid? objectclass: ipaReplicationTopologySegment left: master-X right: master-Y direction: both || left-right || right-left (|| none ?) If we have other special types we change direction accordingly or add another attribute. We already have the list of servers, so we need to add only the list of connections in the topology view. We may need to amend the servers objects to add additional data in some cases. For example indicate whether it is fully installed or not (on creation the topology plugin would complain the server is disconnected until we create the first segment, but that may actually be a good thing :-) Not sure I grok the fully installed part. A server isn't added as a master until it is actually installed, so a prepared master shouldn't show here. Uhmm you may be right, if we can make this a non-problem, all the better. Next question: How to handle changes directly done in the dse.ldif, if everything should be done by the topology plugin it would have to verify and compare the info in cn=config and in the shared tree at every startup of the directory server, which might be complicated by the fact that the replication plugin might already be started and repl agreemnts are active before the topology plugin is started and could do its work. (plugin starting order and dependencies need to be checked). Why do we care which one starts first ? We can simply change replication agreements at any time, so the fact the replication topology (and therefore agreements) can change after startup should not be an issue. Someone could delete an agreement, or worse, add one we don't know about. Does that matter? Someone can do this at any time after startup, so we already need to handle this, why should it be a problem ? It shouldn't be a problem for replication, since everything is dynamic. However I agree we want to avoid churn, so to answer to Ludwig as well I guess we just want to make sure the topology plugin always starts before the replication plugin and amends replication agreements accordingly. +1 - I think this will avoid some problems. What happens to values in the mapping tree that aren't represented in our own topology view? I think we should ignore them if they reference a machine that is not a recognized master, I guess the main issue here is a case when a master got deleted and somehow the cn=config entry was not and we end up with an orphan agreement that the topology plugin initially created but does not recognize as its own. I see 2 options here: 1) We ignore it, and let the admin deal with the issue 2) We mark agreements with a special attribute that indicates they have been generated by the topology plugin, so the plugin can delete any it does not recognize as currently valid. The only problem here is initial migration, but that is not a huge issue
Re: [Freeipa-devel] Move replication topology to the shared tree
On Mon, 2014-06-02 at 12:02 -0600, Rich Megginson wrote: > Why not (selectively) replicate cn=config? Because we do not want to just replicate cn=config (which would be pointless anyway given each server has a completely different set of replication agreements). We want a much more abstract view that the topology plugin can then translate in replication agreements. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/02/2014 02:46 AM, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree There has been some discussion in comments in the ticket, but I'd like to open the discussion to a wider audience to get an agreement on what should be implemented, before writing a design spec. The implementation requires a new IPA plugin for 389 DS and eventually an enhancement of the 389 replication plugin (which depends on some decisions below). In the following I will use the terms “topology plugin” for the new plugin and “replication plugin” for the existing 389 multimaster replication plugin. Lets start with the requirements: What should be achieved by this RFE ? In my opinion there are three different levels of features to implement with this request - providing all replication configuration information consistent across all deployed servers on all servers, eg to easily visualize the replication topology. - Allowing to do sanity checks on replication configuration, denying modifications which would break replication topology or issue warnings. - Use the information in the shared tree to trigger changes to the replication configuration in the corresponding servers, this means to allow to completely control replication configuration with modifications of entries in the shared tree The main questions are 1] which information is needed in the shared tree (eg what params in the repl config should be modifiable) 2] how is the information organized and stored (layout of the repl config information shared tree) 3] how is the interaction of the info in the shared tree and configuration in cn=config and the interaction between the topology plugin and the replication plugin I apologize that I have not yet finished reading through all of this thread and the comments/replies, so perhaps my following comment is out of line: Why not (selectively) replicate cn=config? We keep moving more and more stuff out of cn=config and into the main tree (dna, automember, etc.), to work around the problem that data underneath cn=config is not replicated. We already have customers who have asked for things like database configuration, index configuration, suffix configuration, and many other configurations, to be replicated. And, for a bonus, if we do this right, we might be able to leverage this work to do "real" schema replication. I will note that openldap syncrepl does allow cn=config to be replicated. ad 1] to verify the topology, eg connectivity info about all existing replication agreements is needed, the replication agreements only contain info about the target, and the parameters for connection to the target, but not about the origin. If the data have to evaluated on any server, information about the origin has to be added, eg replicaID, serverID,... In addition, if agreement config has to be changed based on the shared tree all required parameters need to be present, eg replicatedAttributeList, strippedAttrs, replicationEnabled, . Replication agreements only provide information on connections where replication is configured, if connectivity is to be checked independent info about all deployed serevers/replicas is needed. If topology should be validated, do we need params definig requirements, eg each replica to be connected to 1,2,3,... others, type of topology (ring, mesh, star,.) ? ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. In my opinion a 1:1 relation is straight forward, easy to handle and easy to extend (not the full data of a repl agreement need to be present, other attributes are possible). The downside may be a larger number of entries, but this is no problem for the directory server and replication and the utilities eg to visualize a topology will handle this. If the number of entries should be reduced information on multiple replication agreements would have to be stored in one entry, and the problem arises ho to group data belonging to one agreement. LDAP does not provide a simple way to group attribute values in one entry, so all the info related to one agreement (origin, target, replicated attrs and other repl configuration info) could be stored in a single attribute, which will make the attribute as nicely readable and managable as acis. If topology verification and connectivity check is an integral part of the feature, I think a connection oriented view is not sufficient, it might be incomplete, so a server view is required, the server entry would then have the connection information as subentries or as attributes. Ad 3] The replication configuration is stored under cn=config and can be modified either by ldap operations or by
Re: [Freeipa-devel] Move replication topology to the shared tree
On Mon, 2014-06-02 at 16:53 +0200, thierry bordaz wrote:
> If server parameters are changed (port number, repl_admin or DM
> password) will it trigger the corresponding modification on the
> shared
> tree on the remote server ?
Well in the IPA case we do not have these issues as those parameters
either do not change or are not used.
IE, port does not change, it is always the same for all servers
repl admin does not change. DM password is not used (we use GSSAPI for
auth).
I guess we could add optional parameters like:
left-port: 12345
left-admin: cn=foo
right-password: ${SHA-256}ERGSERGDFSGDSF
right-auth: PLAIN
or similar, but I would defer adding 'non-standard' knobs to a later
version. I think v1 should focus on the minimal set of options we need
to make it work well in IPA. I want us to concentrate on getting the
'topology' part right and awesome, not bog ourselves down in
'compatibility' issues to broaden the usability of it.
Once we get it working great within the IPA scope then we can try to
make it more generally usable adding support for corner cases and so on.
Keep in mind if you have special needs you will always be able to
manually add replication agreements that are not managed by the plugin.
Perhaps we can even add an attribute like: topology-exception: TRUE so
that the topology plugin will explicitly avoid touching it even if it
normally would be matching the normal topology criteria and the topology
plugin would normally try to own it.
Simo.
--
Simo Sorce * Red Hat, Inc * New York
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/02/2014 10:46 AM, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree There has been some discussion in comments in the ticket, but I'd like to open the discussion to a wider audience to get an agreement on what should be implemented, before writing a design spec. The implementation requires a new IPA plugin for 389 DS and eventually an enhancement of the 389 replication plugin (which depends on some decisions below). In the following I will use the terms “topology plugin” for the new plugin and “replication plugin” for the existing 389 multimaster replication plugin. Lets start with the requirements: What should be achieved by this RFE ? In my opinion there are three different levels of features to implement with this request - providing all replication configuration information consistent across all deployed servers on all servers, eg to easily visualize the replication topology. - Allowing to do sanity checks on replication configuration, denying modifications which would break replication topology or issue warnings. - Use the information in the shared tree to trigger changes to the replication configuration in the corresponding servers, this means to allow to completely control replication configuration with modifications of entries in the shared tree The main questions are 1] which information is needed in the shared tree (eg what params in the repl config should be modifiable) 2] how is the information organized and stored (layout of the repl config information shared tree) 3] how is the interaction of the info in the shared tree and configuration in cn=config and the interaction between the topology plugin and the replication plugin ad 1] to verify the topology, eg connectivity info about all existing replication agreements is needed, the replication agreements only contain info about the target, and the parameters for connection to the target, but not about the origin. If the data have to evaluated on any server, information about the origin has to be added, eg replicaID, serverID,... In addition, if agreement config has to be changed based on the shared tree all required parameters need to be present, eg replicatedAttributeList, strippedAttrs, replicationEnabled, . Replication agreements only provide information on connections where replication is configured, if connectivity is to be checked independent info about all deployed serevers/replicas is needed. If topology should be validated, do we need params definig requirements, eg each replica to be connected to 1,2,3,... others, type of topology (ring, mesh, star,.) ? ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. In my opinion a 1:1 relation is straight forward, easy to handle and easy to extend (not the full data of a repl agreement need to be present, other attributes are possible). The downside may be a larger number of entries, but this is no problem for the directory server and replication and the utilities eg to visualize a topology will handle this. If the number of entries should be reduced information on multiple replication agreements would have to be stored in one entry, and the problem arises ho to group data belonging to one agreement. LDAP does not provide a simple way to group attribute values in one entry, so all the info related to one agreement (origin, target, replicated attrs and other repl configuration info) could be stored in a single attribute, which will make the attribute as nicely readable and managable as acis. If topology verification and connectivity check is an integral part of the feature, I think a connection oriented view is not sufficient, it might be incomplete, so a server view is required, the server entry would then have the connection information as subentries or as attributes. Ad 3] The replication configuration is stored under cn=config and can be modified either by ldap operations or by editing the dse.ldif. With the topology plugin another source of configuration changes comes into play. The first question is: which information has precendence ? I think if there is info in the shared tree it should be used, and the information in cn=config should be updated. This also means that the topology plugin needs to intercept all mods to the entries in cn=config and have them ignored and handle all updates to the shared tree and trigger changes to the cn=config entries, which then would trigger rebuilds of the in memory replication objects. If server parameters are changed (port number, repl_admin or DM password) will it trigger the corresponding modification on the shared tree on the remote server ? Next qu
Re: [Freeipa-devel] Move replication topology to the shared tree
On Mon, 2014-06-02 at 10:08 -0400, Rob Crittenden wrote: > Simo Sorce wrote: > > However we may want to be able to mark a topology for 'multiple' sets. > > For example we may want to have by default the same topology both for > > the main database and for the CA database. > > I think we should store them separately and making them the "same" would > be applied by a tool, but the data would just reflect the connections. > > I was thinking the object DN would contain the LDAP database name (or > some rough equivalent), so we would store the IPA connections separate > from the CA connections. Ok, we can debate about this, maybe we simply have a flag in the framework that 'links' two topologies and simply replicates any change from one to the other. The only reason I had to 'mark' stuff in a single topology in order to share it is that this way any change is atomic and the 2 topologies cannot diverge, as the objects are the same, if we think the chance of divergence is low or that it is not important because the topology plugin will always prevent disconnected states anyway then we may avoid it, and let the framework try to keep topologies in sync and just loudly warn if they somehow get out of sync (which will happen briefly every time replication of the topology objects happens :). > >> ad 2] the data required are available in the replicationAgreement (and > >> eventually replica) entries, but the question is if there should be a > >> 1:1 relationship to entries in the shared tree or a condensed > >> representation, if there should be a server or connection oriented view. > > > > My answer is no, we need only one object per connection, but config > > entries are per direction (and different ones on different servers). > > We also need to store the type, MMR, read-only, etc, for future-proofing. Store where ? > One entry per connection would mirror what we have now in the mapping > tree (which is generally ok). I wonder if this would be limiting with > other agreement types depending on the schema we use. My idea is that on the connection object you have a set of attributes that tells you how replication happen. So normally you'll have: dn: uuid? objectclass: ipaReplicationTopologySegment left: master-X right: master-Y direction: both || left-right || right-left (|| none ?) If we have other special types we change direction accordingly or add another attribute. > > We already have the list of servers, so we need to add only the list of > > connections in the topology view. We may need to amend the servers > > objects to add additional data in some cases. For example indicate > > whether it is fully installed or not (on creation the topology plugin > > would complain the server is disconnected until we create the first > > segment, but that may actually be a good thing :-) > > Not sure I grok the fully installed part. A server isn't added as a > master until it is actually installed, so a prepared master shouldn't > show here. Uhmm you may be right, if we can make this a non-problem, all the better. > >> Next question: How to handle changes directly done in the dse.ldif, if > >> everything should be done by the topology plugin it would have to verify > >> and compare the info in cn=config and in the shared tree at every > >> startup of the directory server, which might be complicated by the fact > >> that the replication plugin might already be started and repl agreemnts > >> are active before the topology plugin is started and could do its work. > >> (plugin starting order and dependencies need to be checked). > > > > Why do we care which one starts first ? > > We can simply change replication agreements at any time, so the fact the > > replication topology (and therefore agreements) can change after startup > > should not be an issue. > > Someone could delete an agreement, or worse, add one we don't know > about. Does that matter? Someone can do this at any time after startup, so we already need to handle this, why should it be a problem ? However I agree we want to avoid churn, so to answer to Ludwig as well I guess we just want to make sure the topology plugin always starts before the replication plugin and amends replication agreements accordingly. > What happens to values in the mapping tree that aren't represented in > our own topology view? I think we should ignore them if they reference a machine that is not a recognized master, I guess the main issue here is a case when a master got deleted and somehow the cn=config entry was not and we end up with an orphan agreement that the topology plugin initially created but does not recognize as its own. I see 2 options here: 1) We ignore it, and let the admin deal with the issue 2) We mark agreements with a special attribute that indicates they have been generated by the topology plugin, so the plugin can delete any it does not recognize as currently valid. The only problem here is initial migration, but that is not a huge issue
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/02/2014 04:08 PM, Rob Crittenden wrote: Simo Sorce wrote: First of all, very good summary, thanks a lot! Replies in line. On Mon, 2014-06-02 at 10:46 +0200, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree There has been some discussion in comments in the ticket, but I'd like to open the discussion to a wider audience to get an agreement on what should be implemented, before writing a design spec. The implementation requires a new IPA plugin for 389 DS and eventually an enhancement of the 389 replication plugin (which depends on some decisions below). In the following I will use the terms “topology plugin” for the new plugin and “replication plugin” for the existing 389 multimaster replication plugin. Lets start with the requirements: What should be achieved by this RFE ? In my opinion there are three different levels of features to implement with this request - providing all replication configuration information consistent across all deployed servers on all servers, eg to easily visualize the replication topology. - Allowing to do sanity checks on replication configuration, denying modifications which would break replication topology or issue warnings. - Use the information in the shared tree to trigger changes to the replication configuration in the corresponding servers, this means to allow to completely control replication configuration with modifications of entries in the shared tree The main questions are 1] which information is needed in the shared tree (eg what params in the repl config should be modifiable) 2] how is the information organized and stored (layout of the repl config information shared tree) 3] how is the interaction of the info in the shared tree and configuration in cn=config and the interaction between the topology plugin and the replication plugin ad 1] to verify the topology, eg connectivity info about all existing replication agreements is needed, the replication agreements only contain info about the target, and the parameters for connection to the target, but not about the origin. If the data have to evaluated on any server, information about the origin has to be added, eg replicaID, serverID,... In addition, if agreement config has to be changed based on the shared tree all required parameters need to be present, eg replicatedAttributeList, strippedAttrs, replicationEnabled, . Replication agreements only provide information on connections where replication is configured, if connectivity is to be checked independent info about all deployed serevers/replicas is needed. If topology should be validated, do we need params definig requirements, eg each replica to be connected to 1,2,3,... others, type of topology (ring, mesh, star,.) ? Ok from a topology point of view you need the same elements you need to define a graph. That is: nodes and segments. We already have the list of masters in the cn=etc tree, so all we need is to add segments (ie connection objects). As for parameters my idea is that we have a general set of parameters (eg. replicatedAttributeList, strippedAttrs) in the general topology configuration, then we might override them on a per-connection basis if needed (should be very rare). Also note we may need multiple topology sets, because we may have to distinguish between the replication topology for the main shared tree and the replication topology for other databases. However we may want to be able to mark a topology for 'multiple' sets. For example we may want to have by default the same topology both for the main database and for the CA database. I think we should store them separately and making them the "same" would be applied by a tool, but the data would just reflect the connections. I was thinking the object DN would contain the LDAP database name (or some rough equivalent), so we would store the IPA connections separate from the CA connections. ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. My answer is no, we need only one object per connection, but config entries are per direction (and different ones on different servers). We also need to store the type, MMR, read-only, etc, for future-proofing. One entry per connection would mirror what we have now in the mapping tree (which is generally ok). I wonder if this would be limiting with other agreement types depending on the schema we use. In the mapping tree you have a connection as a replicationagreement, but if I understand Simo correctly he wants on object per segment. If two servers are connected this could be one- or bidirectional In my opinion a 1:1 relation is straight forward, easy to handle and easy to extend (not the full data of a repl agreement need to be pr
Re: [Freeipa-devel] Move replication topology to the shared tree
Simo Sorce wrote: > First of all, very good summary, thanks a lot! > Replies in line. > > On Mon, 2014-06-02 at 10:46 +0200, Ludwig Krispenz wrote: >> Ticket 4302 is a request for an enhancement: Move replication topology >> to the shared tree >> >> >> There has been some discussion in comments in the ticket, but I'd like >> to open the discussion to a wider audience to get an agreement on what >> should be implemented, before writing a design spec. >> >> The implementation requires a new IPA plugin for 389 DS and eventually >> an enhancement of the 389 replication plugin (which depends on some >> decisions below). In the following I will use the terms “topology >> plugin” for the new plugin and “replication plugin” for the existing 389 >> multimaster replication plugin. >> >> >> Lets start with the requirements: What should be achieved by this RFE ? >> >> In my opinion there are three different levels of features to implement >> with this request >> >> - providing all replication configuration information consistent across >> all deployed servers on all servers, eg to easily visualize the >> replication topology. >> >> - Allowing to do sanity checks on replication configuration, denying >> modifications which would break replication topology or issue warnings. >> >> - Use the information in the shared tree to trigger changes to the >> replication configuration in the corresponding servers, this means to >> allow to completely control replication configuration with modifications >> of entries in the shared tree >> >> >> The main questions are >> >> 1] which information is needed in the shared tree (eg what params in the >> repl config should be modifiable) >> >> 2] how is the information organized and stored (layout of the repl >> config information shared tree) >> >> 3] how is the interaction of the info in the shared tree and >> configuration in cn=config and the interaction between the topology >> plugin and the replication plugin >> >> >> ad 1] to verify the topology, eg connectivity info about all existing >> replication agreements is needed, the replication agreements only >> contain info about the target, and the parameters for connection to the >> target, but not about the origin. If the data have to evaluated on any >> server, information about the origin has to be added, eg replicaID, >> serverID,... >> >> In addition, if agreement config has to be changed based on the shared >> tree all required parameters need to be present, eg >> replicatedAttributeList, strippedAttrs, replicationEnabled, . >> >> Replication agreements only provide information on connections where >> replication is configured, if connectivity is to be checked independent >> info about all deployed serevers/replicas is needed. >> >> If topology should be validated, do we need params definig requirements, >> eg each replica to be connected to 1,2,3,... others, type of topology >> (ring, mesh, star,.) ? > > Ok from a topology point of view you need the same elements you need to > define a graph. That is: nodes and segments. > > We already have the list of masters in the cn=etc tree, so all we need > is to add segments (ie connection objects). > > As for parameters my idea is that we have a general set of parameters > (eg. replicatedAttributeList, strippedAttrs) in the general topology > configuration, then we might override them on a per-connection basis if > needed (should be very rare). > > Also note we may need multiple topology sets, because we may have to > distinguish between the replication topology for the main shared tree > and the replication topology for other databases. > > However we may want to be able to mark a topology for 'multiple' sets. > For example we may want to have by default the same topology both for > the main database and for the CA database. I think we should store them separately and making them the "same" would be applied by a tool, but the data would just reflect the connections. I was thinking the object DN would contain the LDAP database name (or some rough equivalent), so we would store the IPA connections separate from the CA connections. >> ad 2] the data required are available in the replicationAgreement (and >> eventually replica) entries, but the question is if there should be a >> 1:1 relationship to entries in the shared tree or a condensed >> representation, if there should be a server or connection oriented view. > > My answer is no, we need only one object per connection, but config > entries are per direction (and different ones on different servers). We also need to store the type, MMR, read-only, etc, for future-proofing. One entry per connection would mirror what we have now in the mapping tree (which is generally ok). I wonder if this would be limiting with other agreement types depending on the schema we use. >> In my opinion a 1:1 relation is straight forward, easy to handle and >> easy to extend (not the full data of a repl agre
Re: [Freeipa-devel] Move replication topology to the shared tree
On 06/02/2014 02:59 PM, Simo Sorce wrote: First of all, very good summary, thanks a lot! Replies in line. On Mon, 2014-06-02 at 10:46 +0200, Ludwig Krispenz wrote: Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree There has been some discussion in comments in the ticket, but I'd like to open the discussion to a wider audience to get an agreement on what should be implemented, before writing a design spec. The implementation requires a new IPA plugin for 389 DS and eventually an enhancement of the 389 replication plugin (which depends on some decisions below). In the following I will use the terms “topology plugin” for the new plugin and “replication plugin” for the existing 389 multimaster replication plugin. Lets start with the requirements: What should be achieved by this RFE ? In my opinion there are three different levels of features to implement with this request - providing all replication configuration information consistent across all deployed servers on all servers, eg to easily visualize the replication topology. - Allowing to do sanity checks on replication configuration, denying modifications which would break replication topology or issue warnings. - Use the information in the shared tree to trigger changes to the replication configuration in the corresponding servers, this means to allow to completely control replication configuration with modifications of entries in the shared tree The main questions are 1] which information is needed in the shared tree (eg what params in the repl config should be modifiable) 2] how is the information organized and stored (layout of the repl config information shared tree) 3] how is the interaction of the info in the shared tree and configuration in cn=config and the interaction between the topology plugin and the replication plugin ad 1] to verify the topology, eg connectivity info about all existing replication agreements is needed, the replication agreements only contain info about the target, and the parameters for connection to the target, but not about the origin. If the data have to evaluated on any server, information about the origin has to be added, eg replicaID, serverID,... In addition, if agreement config has to be changed based on the shared tree all required parameters need to be present, eg replicatedAttributeList, strippedAttrs, replicationEnabled, . Replication agreements only provide information on connections where replication is configured, if connectivity is to be checked independent info about all deployed serevers/replicas is needed. If topology should be validated, do we need params definig requirements, eg each replica to be connected to 1,2,3,... others, type of topology (ring, mesh, star,.) ? Ok from a topology point of view you need the same elements you need to define a graph. That is: nodes and segments. We already have the list of masters in the cn=etc tree, so all we need is to add segments (ie connection objects). ok, was not aware you want to extend existing objects, but that's fine. For the segments, my idea was to represent it by two objects (one for each direction), but just having one entry per segment is also ok for me. I was just afraid you wanted to condense the complete topology into on entry As for parameters my idea is that we have a general set of parameters (eg. replicatedAttributeList, strippedAttrs) in the general topology configuration, then we might override them on a per-connection basis if needed (should be very rare). Also note we may need multiple topology sets, because we may have to distinguish between the replication topology for the main shared tree and the replication topology for other databases. However we may want to be able to mark a topology for 'multiple' sets. For example we may want to have by default the same topology both for the main database and for the CA database. yes, having several levels of configurations, which could then be overwritten per database or per segment sounds good ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. My answer is no, we need only one object per connection, but config entries are per direction (and different ones on different servers). accepted :-) In my opinion a 1:1 relation is straight forward, easy to handle and easy to extend (not the full data of a repl agreement need to be present, other attributes are possible). The downside may be a larger number of entries, but this is no problem for the directory server and replication and the utilities eg to visualize a topology will handle this. We want a more abstract and easy to handle view for the topology plugin, in general. If the number of entries should be reduced information on multiple replica
Re: [Freeipa-devel] Move replication topology to the shared tree
First of all, very good summary, thanks a lot! Replies in line. On Mon, 2014-06-02 at 10:46 +0200, Ludwig Krispenz wrote: > Ticket 4302 is a request for an enhancement: Move replication topology > to the shared tree > > > There has been some discussion in comments in the ticket, but I'd like > to open the discussion to a wider audience to get an agreement on what > should be implemented, before writing a design spec. > > The implementation requires a new IPA plugin for 389 DS and eventually > an enhancement of the 389 replication plugin (which depends on some > decisions below). In the following I will use the terms “topology > plugin” for the new plugin and “replication plugin” for the existing 389 > multimaster replication plugin. > > > Lets start with the requirements: What should be achieved by this RFE ? > > In my opinion there are three different levels of features to implement > with this request > > - providing all replication configuration information consistent across > all deployed servers on all servers, eg to easily visualize the > replication topology. > > - Allowing to do sanity checks on replication configuration, denying > modifications which would break replication topology or issue warnings. > > - Use the information in the shared tree to trigger changes to the > replication configuration in the corresponding servers, this means to > allow to completely control replication configuration with modifications > of entries in the shared tree > > > The main questions are > > 1] which information is needed in the shared tree (eg what params in the > repl config should be modifiable) > > 2] how is the information organized and stored (layout of the repl > config information shared tree) > > 3] how is the interaction of the info in the shared tree and > configuration in cn=config and the interaction between the topology > plugin and the replication plugin > > > ad 1] to verify the topology, eg connectivity info about all existing > replication agreements is needed, the replication agreements only > contain info about the target, and the parameters for connection to the > target, but not about the origin. If the data have to evaluated on any > server, information about the origin has to be added, eg replicaID, > serverID,... > > In addition, if agreement config has to be changed based on the shared > tree all required parameters need to be present, eg > replicatedAttributeList, strippedAttrs, replicationEnabled, . > > Replication agreements only provide information on connections where > replication is configured, if connectivity is to be checked independent > info about all deployed serevers/replicas is needed. > > If topology should be validated, do we need params definig requirements, > eg each replica to be connected to 1,2,3,... others, type of topology > (ring, mesh, star,.) ? Ok from a topology point of view you need the same elements you need to define a graph. That is: nodes and segments. We already have the list of masters in the cn=etc tree, so all we need is to add segments (ie connection objects). As for parameters my idea is that we have a general set of parameters (eg. replicatedAttributeList, strippedAttrs) in the general topology configuration, then we might override them on a per-connection basis if needed (should be very rare). Also note we may need multiple topology sets, because we may have to distinguish between the replication topology for the main shared tree and the replication topology for other databases. However we may want to be able to mark a topology for 'multiple' sets. For example we may want to have by default the same topology both for the main database and for the CA database. > ad 2] the data required are available in the replicationAgreement (and > eventually replica) entries, but the question is if there should be a > 1:1 relationship to entries in the shared tree or a condensed > representation, if there should be a server or connection oriented view. My answer is no, we need only one object per connection, but config entries are per direction (and different ones on different servers). > In my opinion a 1:1 relation is straight forward, easy to handle and > easy to extend (not the full data of a repl agreement need to be > present, other attributes are possible). The downside may be a larger > number of entries, but this is no problem for the directory server and > replication and the utilities eg to visualize a topology will handle this. We want a more abstract and easy to handle view for the topology plugin, in general. > If the number of entries should be reduced information on multiple > replication agreements would have to be stored in one entry, and the > problem arises ho to group data belonging to one agreement. LDAP does > not provide a simple way to group attribute values in one entry, so all > the info related to one agreement (origin, target, replicated attrs and > other r
[Freeipa-devel] Move replication topology to the shared tree
Ticket 4302 is a request for an enhancement: Move replication topology to the shared tree There has been some discussion in comments in the ticket, but I'd like to open the discussion to a wider audience to get an agreement on what should be implemented, before writing a design spec. The implementation requires a new IPA plugin for 389 DS and eventually an enhancement of the 389 replication plugin (which depends on some decisions below). In the following I will use the terms “topology plugin” for the new plugin and “replication plugin” for the existing 389 multimaster replication plugin. Lets start with the requirements: What should be achieved by this RFE ? In my opinion there are three different levels of features to implement with this request - providing all replication configuration information consistent across all deployed servers on all servers, eg to easily visualize the replication topology. - Allowing to do sanity checks on replication configuration, denying modifications which would break replication topology or issue warnings. - Use the information in the shared tree to trigger changes to the replication configuration in the corresponding servers, this means to allow to completely control replication configuration with modifications of entries in the shared tree The main questions are 1] which information is needed in the shared tree (eg what params in the repl config should be modifiable) 2] how is the information organized and stored (layout of the repl config information shared tree) 3] how is the interaction of the info in the shared tree and configuration in cn=config and the interaction between the topology plugin and the replication plugin ad 1] to verify the topology, eg connectivity info about all existing replication agreements is needed, the replication agreements only contain info about the target, and the parameters for connection to the target, but not about the origin. If the data have to evaluated on any server, information about the origin has to be added, eg replicaID, serverID,... In addition, if agreement config has to be changed based on the shared tree all required parameters need to be present, eg replicatedAttributeList, strippedAttrs, replicationEnabled, . Replication agreements only provide information on connections where replication is configured, if connectivity is to be checked independent info about all deployed serevers/replicas is needed. If topology should be validated, do we need params definig requirements, eg each replica to be connected to 1,2,3,... others, type of topology (ring, mesh, star,.) ? ad 2] the data required are available in the replicationAgreement (and eventually replica) entries, but the question is if there should be a 1:1 relationship to entries in the shared tree or a condensed representation, if there should be a server or connection oriented view. In my opinion a 1:1 relation is straight forward, easy to handle and easy to extend (not the full data of a repl agreement need to be present, other attributes are possible). The downside may be a larger number of entries, but this is no problem for the directory server and replication and the utilities eg to visualize a topology will handle this. If the number of entries should be reduced information on multiple replication agreements would have to be stored in one entry, and the problem arises ho to group data belonging to one agreement. LDAP does not provide a simple way to group attribute values in one entry, so all the info related to one agreement (origin, target, replicated attrs and other repl configuration info) could be stored in a single attribute, which will make the attribute as nicely readable and managable as acis. If topology verification and connectivity check is an integral part of the feature, I think a connection oriented view is not sufficient, it might be incomplete, so a server view is required, the server entry would then have the connection information as subentries or as attributes. Ad 3] The replication configuration is stored under cn=config and can be modified either by ldap operations or by editing the dse.ldif. With the topology plugin another source of configuration changes comes into play. The first question is: which information has precendence ? I think if there is info in the shared tree it should be used, and the information in cn=config should be updated. This also means that the topology plugin needs to intercept all mods to the entries in cn=config and have them ignored and handle all updates to the shared tree and trigger changes to the cn=config entries, which then would trigger rebuilds of the in memory replication objects. Next question: How to handle changes directly done in the dse.ldif, if everything should be done by the topology plugin it would have to verify and compare the info in cn=config and in the shared tree at every startup of the
