Re: [Freeipa-devel] topology management question
On Fri, 09 Jan 2015 16:07:21 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/09/2015 03:50 PM, Simo Sorce wrote: On Fri, 09 Jan 2015 15:29:02 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:35 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:23:08 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. While this is true, it is also a rare operation. I do not see it as a big deal to be honest. However if you prefer to add a flag attribute that is fine by me too. after thinking a bit more about it, I don't think we need the mark at all. We discussed this already and we came to the conclusion we need it :) ok, I think in the discussion we came up with the mark was with my concerns about removing a replica, and you more or less destroyed my concerns. I agree there could be some corner cases like the ones you sketch below. But it will still be difficult to do the right thing in all cases. Take your example of a marked agreement and no segment. we could arrive at this state in two different scenarios: 1] - take a backup of the shared database - add a segment, a RA will be created and marked - the database gets corrupted and you restore the backup - the agmt is marked and no segment exists - should we really delete the agmt ? Yes. The shared tree is the authoritative repository. If you restore you are required to do due diligence and check what was lost and restore that too. 2] - have a segment and a marked agreement - take a backup of the dse.ldif - delete the segment, agmt is removed - restore the dse.ldif, - the agmt is marked, no segment exists Yes, this is the scenario I was hinting at, it the agreement is marked we know it is not something we need to import (it was already imported) so we just go on and delete it. The agreement would have been marked in two scenarios - the agreement exists and the dom level is raised, so that a segment is created from the agreement - the dom level is set, the plugin active and a segment is addded to the shared tree so that a replication agreement is generated. In all cases where an agreement is marked, there is a 1:1 corresponding segment, so the existence of a segment should be marking enough. I will make the mark_agreement and check_mark as noops, so if we really run into a scenario where a mark would be required, it can be added in one of the methods discussed so far. I recall the problems came in corner cases. Like when a replica starts and finds an agreement. If it is marked and a segment does not exist, we
Re: [Freeipa-devel] topology management question
On 01/09/2015 04:18 PM, Simo Sorce wrote: On Fri, 09 Jan 2015 16:07:21 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/09/2015 03:50 PM, Simo Sorce wrote: On Fri, 09 Jan 2015 15:29:02 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:35 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:23:08 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. While this is true, it is also a rare operation. I do not see it as a big deal to be honest. However if you prefer to add a flag attribute that is fine by me too. after thinking a bit more about it, I don't think we need the mark at all. We discussed this already and we came to the conclusion we need it :) ok, I think in the discussion we came up with the mark was with my concerns about removing a replica, and you more or less destroyed my concerns. I agree there could be some corner cases like the ones you sketch below. But it will still be difficult to do the right thing in all cases. Take your example of a marked agreement and no segment. we could arrive at this state in two different scenarios: 1] - take a backup of the shared database - add a segment, a RA will be created and marked - the database gets corrupted and you restore the backup - the agmt is marked and no segment exists - should we really delete the agmt ? Yes. The shared tree is the authoritative repository. If you restore you are required to do due diligence and check what was lost and restore that too. ok, if we agree on this, we need the mark , convinced again:-) 2] - have a segment and a marked agreement - take a backup of the dse.ldif - delete the segment, agmt is removed - restore the dse.ldif, - the agmt is marked, no segment exists Yes, this is the scenario I was hinting at, it the agreement is marked we know it is not something we need to import (it was already imported) so we just go on and delete it. The agreement would have been marked in two scenarios - the agreement exists and the dom level is raised, so that a segment is created from the agreement - the dom level is set, the plugin active and a segment is addded to the shared tree so that a replication agreement is generated. In all cases where an agreement is marked, there is a 1:1 corresponding segment, so the existence of a segment should be marking enough. I will make the mark_agreement and check_mark as noops, so if we really run into a scenario where a mark would be required, it can be added in one of the methods discussed so far. I recall the problems came in corner cases. Like when a replica starts and finds an agreement. If it is marked and a segment does not exist, we simply delete it. However if it not marked and a
Re: [Freeipa-devel] topology management question
On 01/07/2015 05:35 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:23:08 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. While this is true, it is also a rare operation. I do not see it as a big deal to be honest. However if you prefer to add a flag attribute that is fine by me too. after thinking a bit more about it, I don't think we need the mark at all. The agreement would have been marked in two scenarios - the agreement exists and the dom level is raised, so that a segment is created from the agreement - the dom level is set, the plugin active and a segment is addded to the shared tree so that a replication agreement is generated. In all cases where an agreement is marked, there is a 1:1 corresponding segment, so the existence of a segment should be marking enough. I will make the mark_agreement and check_mark as noops, so if we really run into a scenario where a mark would be required, it can be added in one of the methods discussed so far. Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On Fri, 09 Jan 2015 15:29:02 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:35 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:23:08 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. While this is true, it is also a rare operation. I do not see it as a big deal to be honest. However if you prefer to add a flag attribute that is fine by me too. after thinking a bit more about it, I don't think we need the mark at all. We discussed this already and we came to the conclusion we need it :) The agreement would have been marked in two scenarios - the agreement exists and the dom level is raised, so that a segment is created from the agreement - the dom level is set, the plugin active and a segment is addded to the shared tree so that a replication agreement is generated. In all cases where an agreement is marked, there is a 1:1 corresponding segment, so the existence of a segment should be marking enough. I will make the mark_agreement and check_mark as noops, so if we really run into a scenario where a mark would be required, it can be added in one of the methods discussed so far. I recall the problems came in corner cases. Like when a replica starts and finds an agreement. If it is marked and a segment does not exist, we simply delete it. However if it not marked and a segment does not exist, what do you do ? Without markings you can only import it, and this way you may end up reviving deleted agreements by mistake. The reason this may happen is that cn=config and the main database are separate, and a backup dse.ldif may be use upon start even though the shared database had been already updated, so we cannot count on the replica starting and finding the segment-to-be-deleted-via-replication still there. The marking was needed to make sure that once an agreement was imported we wouldn't do it again later on. I still maintain that marking it is safer and will lead to less issues with phantom segments coming back to life. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On 01/09/2015 03:50 PM, Simo Sorce wrote: On Fri, 09 Jan 2015 15:29:02 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:35 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:23:08 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. While this is true, it is also a rare operation. I do not see it as a big deal to be honest. However if you prefer to add a flag attribute that is fine by me too. after thinking a bit more about it, I don't think we need the mark at all. We discussed this already and we came to the conclusion we need it :) ok, I think in the discussion we came up with the mark was with my concerns about removing a replica, and you more or less destroyed my concerns. I agree there could be some corner cases like the ones you sketch below. But it will still be difficult to do the right thing in all cases. Take your example of a marked agreement and no segment. we could arrive at this state in two different scenarios: 1] - take a backup of the shared database - add a segment, a RA will be created and marked - the database gets corrupted and you restore the backup - the agmt is marked and no segment exists - should we really delete the agmt ? 2] - have a segment and a marked agreement - take a backup of the dse.ldif - delete the segment, agmt is removed - restore the dse.ldif, - the agmt is marked, no segment exists The agreement would have been marked in two scenarios - the agreement exists and the dom level is raised, so that a segment is created from the agreement - the dom level is set, the plugin active and a segment is addded to the shared tree so that a replication agreement is generated. In all cases where an agreement is marked, there is a 1:1 corresponding segment, so the existence of a segment should be marking enough. I will make the mark_agreement and check_mark as noops, so if we really run into a scenario where a mark would be required, it can be added in one of the methods discussed so far. I recall the problems came in corner cases. Like when a replica starts and finds an agreement. If it is marked and a segment does not exist, we simply delete it. However if it not marked and a segment does not exist, what do you do ? Without markings you can only import it, and this way you may end up reviving deleted agreements by mistake. The reason this may happen is that cn=config and the main database are separate, and a backup dse.ldif may be use upon start even though the shared database had been already updated, so we cannot count on the replica starting and finding the segment-to-be-deleted-via-replication still there. The marking was needed to make sure that once an agreement was imported
Re: [Freeipa-devel] topology management question
Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? This way you avoid having to deal with modrdn. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On Wed, 07 Jan 2015 17:23:08 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. While this is true, it is also a rare operation. I do not see it as a big deal to be honest. However if you prefer to add a flag attribute that is fine by me too. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On 01/07/2015 05:13 PM, Simo Sorce wrote: On Wed, 07 Jan 2015 17:11:53 +0100 Ludwig Krispenz lkris...@redhat.com wrote: Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? Simo and I had a discussion on this and had agreed that the marking of a replication agreement as controlled by the plugin could be done by a naming convention on the replication agreements. They are originally created as cn=meToremote host,... and would be renamed to something like cn=local host-to-remote host,. avoiding to add a new attribute to the repl agreement schema. Unfortunately this does not work out of the box. I only discovered afetr implementing and testing (was not aware of before :-) that DS does not implement modrdn operation for internal backends, It just returns unwilling_to_perform. And if it will be implemented the replication plugin will have to be changed as well to catch the mordrdn to update the in memory objects with the new name (which is used in logging). So, if there is no objection, I will go back to the flag solution. What about simply deleting the agreement and adding it back with the new name ? it will stop replication and the restart it again, unnecessary interrupting replication for some time. Assume you have a working topology and then raise the domain level and the plugin becomes active, creates segments and marks agreements as controlled. This should happen as smooth as possible. This way you avoid having to deal with modrdn. Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On 12/05/2014 04:50 PM, Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement Can you explain what you mean if you remove the server first exactly ? What LDAP operation will be performed, by the management tools ? as far as the plugin is concerned a removal of a replica triggers two operations: - removal of the host from the sservers in cn=masters, so the server is no longer considered as managed - removal of the segment(s) connecting the to be removed replica to other still amnaged servers, which should remove the corresponding replication agreements. It was the order of these two operations I was talking - if you remove the segments first, one segment will be the last one connecting this replica to the topology and removal will be rejected We should never remove the segments first indeed. if we can fully control that only specific management tools can be used, we can define the order, but an admin could apply individual operations and still it would be good if nothing breaks Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. internally was probably misleading, but I will think about it again Segments (and their agreements) should be removed as trigger on the master entry getting removed. This should be done even if it causes a split brain, because if the server is removed, no matter how much we wish to keep tropology integrity we effectively are in a split brain situation, keeping toplogy agreements alive w/o the server entry doesn't help. If we can agree on that, that presence/removal of masters is the primary trigger that's fine. I was thinking of situations where a server was removed, but not uninstalled. Just taking it out of the topology, but it could still be reached But there is a problem, which I think is much harder and I am not sure how much effort I should put in resolving it. If we want to have the replication agreements cleaned up after removal of a replica without direct modification of cn=config, we need to follow the path above, but this also means that the last change needs to reach both the removed replica (R) and the last server(S) it is connected to. It would be nice if the changed reached the replica, indeed, but not a big deal if it doesn't, if you are removing the replica it means you are decommissioning it, so it is not really that important that it receives updates, it will be destroyed shortly. That's what I was not sure about, couldn't there be cases where it is not destroyed, just isolated. And if it is not destroyed for whatever reason, it will be removed from the masters group anyway so it will have no permission to replicate back, and no harm is done to the overall domain. The bad thing is that if this change triggers a removal of the replication agreement on S it could be that the change is not replicated to R before the agreement is removed and is lost. There is no way (or no easy) way to know for teh plugin if a change was received by an other server, There is an easy way, contact the other server and see if the change happened in its LDAP tree :) BNut this is not really necessary, as explained above. I was also thinking about some kind of acknowledge mechanism by doing a
Re: [Freeipa-devel] topology management question
On Thu, 11 Dec 2014 14:18:36 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 12/05/2014 04:50 PM, Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement Can you explain what you mean if you remove the server first exactly ? What LDAP operation will be performed, by the management tools ? as far as the plugin is concerned a removal of a replica triggers two operations: - removal of the host from the sservers in cn=masters, so the server is no longer considered as managed - removal of the segment(s) connecting the to be removed replica to other still amnaged servers, which should remove the corresponding replication agreements. It was the order of these two operations I was talking We can define a correct order, the plugin can refuse to do any other order for direct operations (we need to be careful not to refuse replication operations I think). - if you remove the segments first, one segment will be the last one connecting this replica to the topology and removal will be rejected We should never remove the segments first indeed. if we can fully control that only specific management tools can be used, we can define the order, but an admin could apply individual operations and still it would be good if nothing breaks I think we had a plan to return UNWILLING_TO_PERFORM if the admin tries to remove the last segment first. So we would have no problem really, the admin can try and fail. If he wants to remove a master he'll have to remove it from the masters group, and this will trigger the removal of all segments. Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? internally was probably misleading, but I will think about it again Ok, it is important we both understand what issues we see with any of the possible approaches so we can agree on the best one. Segments (and their agreements) should be removed as trigger on the master entry getting removed. This should be done even if it causes a split brain, because if the server is removed, no matter how much we wish to keep tropology integrity we effectively are in a split brain situation, keeping toplogy agreements alive w/o the server entry doesn't help. If we can agree on that, that presence/removal of masters is the primary trigger that's fine. Yes I think we can definitely agree that this is the primary trigger for server removal/addition. I was thinking of situations where a server was removed, but not uninstalled. Understood, but even then it makes no real difference, once the server is removed from the group of masters it will not be able to replicate outbound anymore as the other master's ACIs will not recognize this server credentials as valid replicator creds. Just taking it out of the topology, but it could still be reached It can be reached, and that may be a problem for clients. But in the long term this should be true only for clients manually configured to reach that
Re: [Freeipa-devel] topology management question
On 11.12.2014 15:20, Simo Sorce wrote: On Thu, 11 Dec 2014 14:18:36 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 12/05/2014 04:50 PM, Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement Can you explain what you mean if you remove the server first exactly ? What LDAP operation will be performed, by the management tools ? as far as the plugin is concerned a removal of a replica triggers two operations: - removal of the host from the sservers in cn=masters, so the server is no longer considered as managed - removal of the segment(s) connecting the to be removed replica to other still amnaged servers, which should remove the corresponding replication agreements. It was the order of these two operations I was talking We can define a correct order, the plugin can refuse to do any other order for direct operations (we need to be careful not to refuse replication operations I think). - if you remove the segments first, one segment will be the last one connecting this replica to the topology and removal will be rejected We should never remove the segments first indeed. if we can fully control that only specific management tools can be used, we can define the order, but an admin could apply individual operations and still it would be good if nothing breaks I think we had a plan to return UNWILLING_TO_PERFORM if the admin tries to remove the last segment first. So we would have no problem really, the admin can try and fail. If he wants to remove a master he'll have to remove it from the masters group, and this will trigger the removal of all segments. Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? internally was probably misleading, but I will think about it again Ok, it is important we both understand what issues we see with any of the possible approaches so we can agree on the best one. Segments (and their agreements) should be removed as trigger on the master entry getting removed. This should be done even if it causes a split brain, because if the server is removed, no matter how much we wish to keep tropology integrity we effectively are in a split brain situation, keeping toplogy agreements alive w/o the server entry doesn't help. If we can agree on that, that presence/removal of masters is the primary trigger that's fine. Yes I think we can definitely agree that this is the primary trigger for server removal/addition. I was thinking of situations where a server was removed, but not uninstalled. Understood, but even then it makes no real difference, once the server is removed from the group of masters it will not be able to replicate outbound anymore as the other master's ACIs will not recognize this server credentials as valid replicator creds. Just taking it out of the topology, but it could still be reached It can be reached, and that may be a problem for clients. But in the long term this should be true only for clients manually configured
Re: [Freeipa-devel] topology management question
On Thu, 11 Dec 2014 17:03:55 +0100 Petr Spacek pspa...@redhat.com wrote: On 11.12.2014 15:20, Simo Sorce wrote: On Thu, 11 Dec 2014 14:18:36 +0100 Ludwig Krispenz lkris...@redhat.com wrote: On 12/05/2014 04:50 PM, Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement Can you explain what you mean if you remove the server first exactly ? What LDAP operation will be performed, by the management tools ? as far as the plugin is concerned a removal of a replica triggers two operations: - removal of the host from the sservers in cn=masters, so the server is no longer considered as managed - removal of the segment(s) connecting the to be removed replica to other still amnaged servers, which should remove the corresponding replication agreements. It was the order of these two operations I was talking We can define a correct order, the plugin can refuse to do any other order for direct operations (we need to be careful not to refuse replication operations I think). - if you remove the segments first, one segment will be the last one connecting this replica to the topology and removal will be rejected We should never remove the segments first indeed. if we can fully control that only specific management tools can be used, we can define the order, but an admin could apply individual operations and still it would be good if nothing breaks I think we had a plan to return UNWILLING_TO_PERFORM if the admin tries to remove the last segment first. So we would have no problem really, the admin can try and fail. If he wants to remove a master he'll have to remove it from the masters group, and this will trigger the removal of all segments. Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements Why should we keep it internally ? If you mark the agreements as managed by setting an attribute on them, then you will never have any issue recognizing a managed agreement in cn=config, and you will also immediately find out it is old as it is not backed by a segment so you will safely remove it. I didn't want to add new flags/fields to the replication agreements as long as anything can be handled by the data in the shared tree. We have too. I think it is a must or we will find numerous corner cases. Is there a specific reason why you do not want to add flags to replication agreements in cn=config ? internally was probably misleading, but I will think about it again Ok, it is important we both understand what issues we see with any of the possible approaches so we can agree on the best one. Segments (and their agreements) should be removed as trigger on the master entry getting removed. This should be done even if it causes a split brain, because if the server is removed, no matter how much we wish to keep tropology integrity we effectively are in a split brain situation, keeping toplogy agreements alive w/o the server entry doesn't help. If we can agree on that, that presence/removal of masters is the primary trigger that's fine. Yes I think we can definitely agree that this is the primary trigger for server removal/addition. I was thinking of situations where a server was removed, but not uninstalled. Understood, but even then it makes no real difference, once the server is removed from the group of masters it will not be able to replicate outbound anymore as the other master's ACIs will not recognize this server credentials as valid replicator creds. Just taking it out of
Re: [Freeipa-devel] topology management question
Simo Sorce wrote: On Sun, 07 Dec 2014 10:25:06 -0500 Rob Crittenden rcrit...@redhat.com wrote: Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement I had another, sort of unrelated thought about this, thinking about deleting servers. What happens if a replication conflict entry gets added? This would happen in case a provisioning system tries to instantiate 2 servers with the same name at the same time talking to different existing masters. Sure and quite possible if there is some link down and two admins doing the same thing. While both exist I imagine that the actual agreement would reflect whichever entry is processed last. Probably not the end of the world. But how do you remove the conflict entry without also potentially deleting that master? It should probably delete both, the domain would be pretty messed up anyone and we have no easy way to know which of the 2 is part of the domain and which one has all replication severed due to their keys being overwritten by the other server ones. Yes, this is what I was leaning towards as well, but the plugin may need to specifically do this. It all depends on the search filters Ludwig uses to find the master(s) to operate on. And should this be automatically detected? So in reality the fact that there is a duplicate agreement in topology probably won't hurt anything at all since we don't really define much there. The actual agreement(s) will still be created just fine, but this extra topology record could be confusing depending on whether it gets returned by anything. I guess the only thing we can reasonably do is to make recommendations on how to deal with replicas deployments to avoid this case and instructions on how to remove remnants entries if any. I brought this up in case deleting the conflict entry would create an orphan, for example. It occurs to me that perhaps the conflict entry may not actually contain anything different than the real entry. It isn't like we store a ton of data. So I wonder if one deletes a conflict entry it shouldn't trigger topology changes. But that is likely going to require extra logic. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
On Tue, 09 Dec 2014 09:21:43 -0500 Rob Crittenden rcrit...@redhat.com wrote: Simo Sorce wrote: On Sun, 07 Dec 2014 10:25:06 -0500 Rob Crittenden rcrit...@redhat.com wrote: Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement I had another, sort of unrelated thought about this, thinking about deleting servers. What happens if a replication conflict entry gets added? This would happen in case a provisioning system tries to instantiate 2 servers with the same name at the same time talking to different existing masters. Sure and quite possible if there is some link down and two admins doing the same thing. While both exist I imagine that the actual agreement would reflect whichever entry is processed last. Probably not the end of the world. But how do you remove the conflict entry without also potentially deleting that master? It should probably delete both, the domain would be pretty messed up anyone and we have no easy way to know which of the 2 is part of the domain and which one has all replication severed due to their keys being overwritten by the other server ones. Yes, this is what I was leaning towards as well, but the plugin may need to specifically do this. It all depends on the search filters Ludwig uses to find the master(s) to operate on. And should this be automatically detected? So in reality the fact that there is a duplicate agreement in topology probably won't hurt anything at all since we don't really define much there. The actual agreement(s) will still be created just fine, but this extra topology record could be confusing depending on whether it gets returned by anything. I guess the only thing we can reasonably do is to make recommendations on how to deal with replicas deployments to avoid this case and instructions on how to remove remnants entries if any. I brought this up in case deleting the conflict entry would create an orphan, for example. It occurs to me that perhaps the conflict entry may not actually contain anything different than the real entry. It isn't like we store a ton of data. So I wonder if one deletes a conflict entry it shouldn't trigger topology changes. But that is likely going to require extra logic. I think we can avoid this situation completely going forward by introducing the concept of ephemeral replication master (what a mouthful eh? :). The idea is that when creating a new replica the installer always determines deterministically a server among all to contact and creates the master entry there with an add operation. So if two masters try to be installed they will conflict early and one will fail. If we are in split-brain the one on the wrong side will fail to contact the master and fail. If the agreed master is down, no replicas can be installed until it is restored or removed (or maybe a force flag is used to work around it for whatever reason). I think this could work, but I am not sure how well it will mesh with the new install procedure work that promotes a normal client because then the basic host keytab will be already broken possibly. So perhaps we keep this possible plan in mind (open a ticket) ? But wait a bit to understand what we will end up with on the promotion changes side. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
Simo Sorce wrote: On Thu, 04 Dec 2014 14:33:09 +0100 Ludwig Krispenz lkris...@redhat.com wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement I had another, sort of unrelated thought about this, thinking about deleting servers. What happens if a replication conflict entry gets added? While both exist I imagine that the actual agreement would reflect whichever entry is processed last. Probably not the end of the world. But how do you remove the conflict entry without also potentially deleting that master? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] topology management question
Ludwig Krispenz wrote: hi, I just have another (hopefully this will end soon) issue I want to get your input. (please read to teh end first) To recapture the conditions: - the topology plugin manages the connections between servers as segments in the shared tree - it is authoritative for managed servers, eg it controls all connections between servers listed under cn=masters, it is permissive for connection to other servers - it rejects any removal of a segment, which would disconnect the topology. - a change in topology can be applied to any server in the topology, it will reach the respective servers and the plugin will act upon it Now there is a special case, causing a bit of trouble. If a replica is to be removed from the topology, this means that the replication agreements from and to this replica should be removed, the server should be removed from the manages servers. The problem is that: - if you remove the server first, the server becomes unmanaged and removal of the segment will not trigger a removal of the replication agreement - if you remove the segments first, one segment will be the last one connecting this replica to the topology and removal will be rejected Now, with some effort this can be resolved, eg if the server is removed, keep it internally as removed server and for segments connecting this server trigger removal of replication agreements or mark a the last segment, when tried to remove, as pending and once the server is removed also remove the corresponding repl agreements But there is a problem, which I think is much harder and I am not sure how much effort I should put in resolving it. If we want to have the replication agreements cleaned up after removal of a replica without direct modification of cn=config, we need to follow the path above, but this also means that the last change needs to reach both the removed replica (R) and the last server(S) it is connected to. The bad thing is that if this change triggers a removal of the replication agreement on S it could be that the change is not replicated to R before the agreement is removed and is lost. There is no way (or no easy) way to know for teh plugin if a change was received by an other server, I was also thinking about some kind of acknowledge mechanism by doing a ping pong of changes, but the problem always is the same that one server does not know if the other has received it. And even if this would theoretically work, we cannot be sure that R is not shutdown and only the remaining topology is tried to be cleaned up, so S would wait forever. My suggestion to resolve this (in most cases) is to define a wait interval, after the final combination of removal of a server and its connecting segment is received, wait for some time and then remove the corresponding replication agreements. So I'm asking you if this would be acceptable or if you have a better solution. As I understood the proposal, you receive change requests which are validated and applied on the server that received them. This is going to cause changes to be replicated. On servers that get these replicated changes they simply apply the changes w/o applying the associated logic. Blindly, in other words. Doesn't that make this problem go away? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
