Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do upgrade only if needed not every restart. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? Sorry, I will add more details there, if this is not clear. ipa-upgrateconfig will be removed ipa-ldap-updater will not be able to do overall update, you will need to specify options and update file. 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) I have no strong feelings on this one, but IMO it is always better to have some plan B if we choose to indeed implement some yet
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 10:55 Martin Kosek napsal(a): On 03/03/2015 09:55 AM, Martin Basti wrote: On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. What if we will need some update which needs to execute time-consuming system check during every upgrade in future? User can always run the upgrade manually, with --skip-version-check, and then configuration plugins will decide if the upgrade is needed. I tend to agree with Martin, I would prefer to be on the safe side and not run config upgrades every time, unless we are explicitly asked to or
Re: [Freeipa-devel] [PATCHES 399-401] Allow multiple API instances
On 03/03/2015 04:01 PM, Martin Kosek wrote: On 03/03/2015 03:49 PM, Jan Cholasta wrote: Hi, the attached patches provide an attempt to fix https://fedorahosted.org/freeipa/ticket/3090. Patch 401 serves as an example and modifies ipa-advise to use its own API instance for Advice plugins. Honza Thanks. At least patches 399 and 400 look reasonable short for 4.2. So with these patches, could we also get rid of temporary_ldap2_connection we have in ipa-replica-install? Petr3 may have other examples he met in the past... Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel 401 seems reasonable enough to me too, the bulk of the code is mostly just moving the code around and renaming variables. Plus we have a very extensive (100%) coverage for the advise tool, so I wouldn't exclude it from the patchset. Tomas ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. Yet updates at start are the only option with container
Re: [Freeipa-devel] [PATCHES 399-401] Allow multiple API instances
On 03/03/2015 04:09 PM, Jan Cholasta wrote: Dne 3.3.2015 v 16:04 Tomas Babej napsal(a): On 03/03/2015 04:01 PM, Martin Kosek wrote: On 03/03/2015 03:49 PM, Jan Cholasta wrote: Hi, the attached patches provide an attempt to fix https://fedorahosted.org/freeipa/ticket/3090. Patch 401 serves as an example and modifies ipa-advise to use its own API instance for Advice plugins. Honza Thanks. At least patches 399 and 400 look reasonable short for 4.2. So with these patches, could we also get rid of temporary_ldap2_connection we have in ipa-replica-install? Petr3 may have other examples he met in the past... I think we can. Shall I prepare a patch? If it is reasonable simple, I would go for it. It would be another selling point for your patches. Martin 401 seems reasonable enough to me too, the bulk of the code is mostly just moving the code around and renaming variables. Right. Plus we have a very extensive (100%) coverage for the advise tool, so I wouldn't exclude it from the patchset. +1 Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES 399-401] Allow multiple API instances
Dne 3.3.2015 v 16:04 Tomas Babej napsal(a): On 03/03/2015 04:01 PM, Martin Kosek wrote: On 03/03/2015 03:49 PM, Jan Cholasta wrote: Hi, the attached patches provide an attempt to fix https://fedorahosted.org/freeipa/ticket/3090. Patch 401 serves as an example and modifies ipa-advise to use its own API instance for Advice plugins. Honza Thanks. At least patches 399 and 400 look reasonable short for 4.2. So with these patches, could we also get rid of temporary_ldap2_connection we have in ipa-replica-install? Petr3 may have other examples he met in the past... I think we can. Shall I prepare a patch? Martin 401 seems reasonable enough to me too, the bulk of the code is mostly just moving the code around and renaming variables. Right. Plus we have a very extensive (100%) coverage for the advise tool, so I wouldn't exclude it from the patchset. +1 Tomas -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 16:11, Simo Sorce wrote: On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. I do not like idea of auto upgrading during (re)start as well, it is not expected by user. At least, upgrade takes 5-10 minutes, user may not to know the upgrade is happening. We have
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 04:43 PM, Martin Basti wrote: On 03/03/15 16:11, Simo Sorce wrote: On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. I do not like idea of auto upgrading during (re)start as well, it is
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 16:47, Martin Kosek wrote: On 03/03/2015 04:43 PM, Martin Basti wrote: On 03/03/15 16:11, Simo Sorce wrote: On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. I do not like idea of auto upgrading during (re)start as well, it is not expected by user. At least,
Re: [Freeipa-devel] [PATCHES 399-401] Allow multiple API instances
On 03/03/2015 03:49 PM, Jan Cholasta wrote: Hi, the attached patches provide an attempt to fix https://fedorahosted.org/freeipa/ticket/3090. Patch 401 serves as an example and modifies ipa-advise to use its own API instance for Advice plugins. Honza Thanks. At least patches 399 and 400 look reasonable short for 4.2. So with these patches, could we also get rid of temporary_ldap2_connection we have in ipa-replica-install? Petr3 may have other examples he met in the past... Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 16:10, Martin Kosek wrote: On 03/03/2015 04:04 PM, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. rob Storing the version is something we just have to do, we need it for missing upgrade detection in FedUp case. We also need for container use case, to make sure that we can check whether we
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 3.3.2015 10:58, Martin Kosek wrote: On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Maybe I'm missing something but if not state.get('dns_forward_zones_supported'): # upgrade to forward zones here seems less error-prone than if version 4.0: #
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 11:04 Petr Spacek napsal(a): On 3.3.2015 10:58, Martin Kosek wrote: On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Maybe I'm missing something but if not state.get('dns_forward_zones_supported'): # upgrade to forward zones here seems less error-prone than if version 4.0: # upgrade to forward
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 11:06 AM, Jan Cholasta wrote: Dne 3.3.2015 v 11:04 Petr Spacek napsal(a): On 3.3.2015 10:58, Martin Kosek wrote: On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Maybe I'm missing something but if not
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 11:04, Petr Spacek wrote: On 3.3.2015 10:58, Martin Kosek wrote: On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Maybe I'm missing something but if not state.get('dns_forward_zones_supported'): # upgrade to forward zones here seems less error-prone than if version 4.0: # upgrade to forward zones here # make me
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 12:08 Martin Kosek napsal(a): On 03/03/2015 11:06 AM, Jan Cholasta wrote: Dne 3.3.2015 v 11:04 Petr Spacek napsal(a): On 3.3.2015 10:58, Martin Kosek wrote: On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Maybe I'm missing something but if not state.get('dns_forward_zones_supported'): # upgrade to forward
Re: [Freeipa-devel] [PATCH 0001] ipa-client-install: attempt to get host TGT several times before aborting client installation
Martin Babinsky wrote: On 03/02/2015 04:28 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 01/12/2015 05:45 PM, Martin Babinsky wrote: related to ticket https://fedorahosted.org/freeipa/ticket/4808 this patch seems to be a bit forgotten. It works, looks fine. One minor issue: trailing whitespaces in the man page. I also wonder if it shouldn't be used in other tools which call kinit with keytab: * ipa-client-automount:434 * ipa-client-install:2591 (this usage should be fine since it's used for server installation) * dcerpc.py:545 * rpcserver.py: 971, 981 (armor for web ui forms base auth) Most importantly the ipa-client-automount because it's called from ipa-client-install (if location is specified) and therefore it might fail during client installation. Or also, kinit call with admin creadentials worked for the user but I wonder if it was just a coincidence and may break under slightly different but similar conditions. I think that's a fine idea. In fact there is already a function that could be extended, kinit_hostprincipal(). rob So in principle we could add multiple TGT retries to kinit_hostprincipal() and then plug this function to all the places Petr mentioned in order to provide this functionality each time TGT is requested using keytab. Do I understand it correctly? Honestly I think I'd only do the retries on client installation. I don't know that the other uses would really benefit or need this. But this is an opportunity to consolidate some code, so I guess the approach I'd take is to add an option to kinit_hostprincipal of retries=0 so that only a single kinit is done. The client installers would pass in some value. This change is quite a bit more invasive but it's also early in the release cycle so the risk will be spread out. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. I have no more objections. * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 3.3.2015 11:01, Jan Cholasta wrote: I would very much prefer to do it the other way around, so that most bugs in the code are caught early, instead of hidden behind the version comparison. +1 -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0001] ipa-client-install: attempt to get host TGT several times before aborting client installation
On 03/03/2015 02:32 PM, Rob Crittenden wrote: Martin Babinsky wrote: On 03/02/2015 04:28 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 01/12/2015 05:45 PM, Martin Babinsky wrote: related to ticket https://fedorahosted.org/freeipa/ticket/4808 this patch seems to be a bit forgotten. It works, looks fine. One minor issue: trailing whitespaces in the man page. I also wonder if it shouldn't be used in other tools which call kinit with keytab: * ipa-client-automount:434 * ipa-client-install:2591 (this usage should be fine since it's used for server installation) * dcerpc.py:545 * rpcserver.py: 971, 981 (armor for web ui forms base auth) Most importantly the ipa-client-automount because it's called from ipa-client-install (if location is specified) and therefore it might fail during client installation. Or also, kinit call with admin creadentials worked for the user but I wonder if it was just a coincidence and may break under slightly different but similar conditions. I think that's a fine idea. In fact there is already a function that could be extended, kinit_hostprincipal(). rob So in principle we could add multiple TGT retries to kinit_hostprincipal() and then plug this function to all the places Petr mentioned in order to provide this functionality each time TGT is requested using keytab. Do I understand it correctly? Honestly I think I'd only do the retries on client installation. I don't know that the other uses would really benefit or need this. But this is an opportunity to consolidate some code, so I guess the approach I'd take is to add an option to kinit_hostprincipal of retries=0 so that only a single kinit is done. The client installers would pass in some value. This change is quite a bit more invasive but it's also early in the release cycle so the risk will be spread out. rob Ok I will try to implement these changes and submit them as separate patchset. -- Martin^3 Babinsky ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do upgrade only if needed not every restart. If there is nothing to upgrade, nothing will be upgraded. The effect is the same. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? Sorry, I will add more details there, if this is not clear. ipa-upgrateconfig will be removed ipa-ldap-updater will
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. What if we will need some update which needs to execute time-consuming system check during every upgrade in future? User can always run the upgrade manually, with --skip-version-check, and then configuration plugins will decide if the upgrade is needed. My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do
Re: [Freeipa-devel] One-way trust design
On Tue, 03 Mar 2015, Jan Pazdziora wrote: On Mon, Feb 23, 2015 at 06:02:53PM +0200, Alexander Bokovoy wrote: trust-related functionality would be limited to IPA admins or TDO object in LDAP would have to be more accessible. Given that TDO credentials can be used to compromise access to our domain, it is not Could you clarify which domain is the our domain? From SMB perspective whole IPA realm is a single domain. advisable to give a wider access to them. As a side-effect of reducing exposure of TDO credentials, FreeIPA lost ability to establish and use one-way trust to Active Directory. The Lost ability might be confusing -- was removed in 3.1 (?) might be better. We never had it as a feature so support for that wasn't removed. Rather, we lost ability to add that support. purpose of this feature is to regain the one-way trust support, yet without giving an elevated access to TDO credentials. You might also want to either add a note or a link, explaining why one-way trust is harder than two-way, IOW, why we lost the one-way ability when we have the two-way one. I think current text covers it clearly. If you have concrete suggestions, feel free to edit the wiki, it is not locked down. :) -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 11:05 AM, Jan Cholasta wrote: Dne 3.3.2015 v 10:58 Martin Kosek napsal(a): On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. How exactly is adding another logic layer on top of the configuration update more simple and stupid? Maybe it isn't. I
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. Petr^2 Spacek My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do upgrade only if needed not every restart. If there is nothing to upgrade, nothing will be upgraded. The effect is the same. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which
Re: [Freeipa-devel] One-way trust design
On Mon, Feb 23, 2015 at 06:02:53PM +0200, Alexander Bokovoy wrote: trust-related functionality would be limited to IPA admins or TDO object in LDAP would have to be more accessible. Given that TDO credentials can be used to compromise access to our domain, it is not Could you clarify which domain is the our domain? advisable to give a wider access to them. As a side-effect of reducing exposure of TDO credentials, FreeIPA lost ability to establish and use one-way trust to Active Directory. The Lost ability might be confusing -- was removed in 3.1 (?) might be better. purpose of this feature is to regain the one-way trust support, yet without giving an elevated access to TDO credentials. You might also want to either add a note or a link, explaining why one-way trust is harder than two-way, IOW, why we lost the one-way ability when we have the two-way one. -- Jan Pazdziora Principal Software Engineer, Identity Management Engineering, Red Hat ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 10:58 Martin Kosek napsal(a): On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. How exactly is adding another logic layer on top of the configuration update more simple and stupid? Martin -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 11:00 Martin Basti napsal(a): On 03/03/15 10:55, Jan Cholasta wrote: Dne 3.3.2015 v 09:55 Martin Basti napsal(a): On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: ... 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Right, but what's your point? We are not discussing order of updates here, I'm perfectly fine with schema updates being done before configuration updates. So you want to run schema update before configuration upgrade every restart? OR you want to run schema update if needed based on version, and then run configuration upgrade? I don't really care, as long as the schema is up-to-date, but I guess there is no harm in always running schema update either. -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 09:55 AM, Martin Basti wrote: On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. What if we will need some update which needs to execute time-consuming system check during every upgrade in future? User can always run the upgrade manually, with --skip-version-check, and then configuration plugins will decide if the upgrade is needed. I tend to agree with Martin, I would prefer to be on the safe side and not run config upgrades every time, unless we are explicitly
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 09:55 Martin Basti napsal(a): On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Right, but what's your point? We are not discussing order of updates here, I'm perfectly fine with schema updates being done before configuration updates. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. Yes, then why do we need another one, especially so when it does not provide any significant speed-up? What if we will need some update which needs to execute time-consuming system check during every upgrade in future? Then we deal with the optimization in the future, instead of
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 2.3.2015 18:54, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. I have no more objections. IMHO the problem with long-running RPM upgrade should be approached from the other way: Just print message 'IPA server upgrade is running, press CTRL+C if you want to destroy your IPA server'. bind-dyndb-ldap prints a message about SELinux in RPM scriptlets for couple releases now and nobody complained (yet? :-). Petr^2 Spacek * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 10:55, Jan Cholasta wrote: Dne 3.3.2015 v 09:55 Martin Basti napsal(a): On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Right, but what's your point? We are not discussing order of updates here, I'm perfectly fine with schema updates being done before configuration updates. So you want to run schema update before configuration upgrade every restart? OR you want to run schema update if needed based on version, and then run configuration upgrade? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. Yes, then why do we need another one, especially so when
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Tue, 2015-03-03 at 15:33 +0100, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? It's a hard problem. First of all DS should never deadlock, we shouldn't change all our process because one component has a bug If DS deadlocks, it will do so whether it is run in RPM or not, we need to add some way to terminate the upgrade ourselves if it lasts too long I guess. The problem with an asynchronous restart is ... when do you do that ? What triggers it ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCHES 399-401] Allow multiple API instances
Hi, the attached patches provide an attempt to fix https://fedorahosted.org/freeipa/ticket/3090. Patch 401 serves as an example and modifies ipa-advise to use its own API instance for Advice plugins. Honza -- Jan Cholasta From 3715c9b4ca43eab6c5ad01b34cd1b14838241bde Mon Sep 17 00:00:00 2001 From: Jan Cholasta jchol...@redhat.com Date: Mon, 16 Feb 2015 13:11:38 + Subject: [PATCH 1/3] ipalib: Allow multiple API instances Merged the Registrar class into the Registry class. Plugins are now registered globally instead of in ipalib.api and are instantiated per-API instance. Different set of plugin base classes can be used in each API instance. https://fedorahosted.org/freeipa/ticket/3090 --- ipalib/backend.py | 3 + ipalib/frontend.py| 10 +- ipalib/plugable.py| 204 -- ipatests/test_ipalib/test_plugable.py | 119 4 files changed, 185 insertions(+), 151 deletions(-) diff --git a/ipalib/backend.py b/ipalib/backend.py index 2100589..4c1001d 100644 --- a/ipalib/backend.py +++ b/ipalib/backend.py @@ -27,7 +27,10 @@ import os from errors import PublicError, InternalError, CommandError from request import context, Connection, destroy_context +register = plugable.Registry() + +@register.base() class Backend(plugable.Plugin): Base class for all backend plugins. diff --git a/ipalib/frontend.py b/ipalib/frontend.py index 98070b8..e82a03a 100644 --- a/ipalib/frontend.py +++ b/ipalib/frontend.py @@ -27,7 +27,7 @@ from distutils import version from ipapython.version import API_VERSION from ipapython.ipa_log_manager import root_logger from base import NameSpace -from plugable import Plugin, is_production_mode +from plugable import Plugin, Registry, is_production_mode from parameters import create_param, Param, Str, Flag, Password from output import Output, Entry, ListOfEntries from text import _ @@ -40,6 +40,9 @@ from textwrap import wrap RULE_FLAG = 'validation_rule' +register = Registry() + + def rule(obj): assert not hasattr(obj, RULE_FLAG) setattr(obj, RULE_FLAG, True) @@ -366,6 +369,7 @@ class HasParam(Plugin): setattr(self, name, namespace) +@register.base() class Command(HasParam): A public IPA atomic operation. @@ -1120,6 +1124,7 @@ class Local(Command): return self.forward(*args, **options) +@register.base() class Object(HasParam): finalize_early = False @@ -1278,6 +1283,7 @@ class Attribute(Plugin): super(Attribute, self)._on_finalize() +@register.base() class Method(Attribute, Command): A command with an associated object. @@ -1364,6 +1370,7 @@ class Method(Attribute, Command): yield param +@register.base() class Updater(Method): An LDAP update with an associated object (always update). @@ -1423,6 +1430,7 @@ class _AdviceOutput(object): self.content.append(line) +@register.base() class Advice(Plugin): Base class for advices, plugins for ipa-advise. diff --git a/ipalib/plugable.py b/ipalib/plugable.py index a6504d1..aae7626 100644 --- a/ipalib/plugable.py +++ b/ipalib/plugable.py @@ -74,18 +74,94 @@ class Registry(object): For forward compatibility, make sure that the module-level instance of this object is named register. -# TODO: Instead of auto-loading when plugin modules are imported, -# plugins should be stored in this object. -# The API should examine it and load plugins explicitly. -def __call__(self): -from ipalib import api -def decorator(cls): -api.register(cls) -return cls +__allowed = {} +__registered = set() + +def base(self): +def decorator(base): +if not inspect.isclass(base): +raise TypeError('plugin base must be a class; got %r' % base) + +if base in self.__allowed: +raise errors.PluginDuplicateError(plugin=base) + +self.__allowed[base] = {} + +return base + +return decorator + +def __findbases(self, klass): + +Iterates through allowed bases that ``klass`` is a subclass of. + +Raises `errors.PluginSubclassError` if ``klass`` is not a subclass of +any allowed base. + +:param klass: The plugin class to find bases for. + +found = False +for (base, sub_d) in self.__allowed.iteritems(): +if issubclass(klass, base): +found = True +yield (base, sub_d) +if not found: +raise errors.PluginSubclassError( +plugin=klass, bases=self.__allowed.keys() +) + +def __call__(self, override=False): +def decorator(klass): +if not inspect.isclass(klass): +raise TypeError('plugin must be a class; got %r' % klass) + +# Raise DuplicateError if
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Tue, 2015-03-03 at 15:40 +0100, Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I think this is wrong, sorry. It means unattended installs will just fail to restart and require manual intervention. At the very least this process needs to be conditional, and upgrade needs to be done automatically. If the admin insist he can set something in default.conf to block autoupgrades or something. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 03:49 PM, Simo Sorce wrote: On Tue, 2015-03-03 at 15:33 +0100, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? It's a hard problem. First of all DS should never deadlock, we shouldn't change all our process because one component has a bug If DS deadlocks, it will do so whether it is run in RPM or not, we need to add some way to terminate the upgrade ourselves if it lasts too long I guess. The problem with an asynchronous restart is ... when do you do that ? What triggers it ? systemctl restart ipa.service --no-block :-) If there is no better place for the upgrade outside of RPM transaction, I am fine, as long as ipactl writes the warning for FedUp users, as Martin mentioned, to make sure they do not run un-upgraded IPA. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel