Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do upgrade only if needed not every restart. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? Sorry, I will add more details there, if this is not clear. ipa-upgrateconfig will be removed ipa-ldap-updater will not be able to do overall update, you will need to specify options and update file. 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) I have no strong feelings on this one, but IMO it is always better to have some plan B if we choose to indeed implement some yet
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 10:55 Martin Kosek napsal(a): On 03/03/2015 09:55 AM, Martin Basti wrote: On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. What if we will need some update which needs to execute time-consuming system check during every upgrade in future? User can always run the upgrade manually, with --skip-version-check, and then configuration plugins will decide if the upgrade is needed. I tend to agree with Martin, I would prefer to be on the safe side and not run config upgrades every time, unless we are explicitly asked to or
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. Yet updates at start are the only option with container
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 16:11, Simo Sorce wrote: On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. I do not like idea of auto upgrading during (re)start as well, it is not expected by user. At least, upgrade takes 5-10 minutes, user may not to know the upgrade is happening. We have
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 04:43 PM, Martin Basti wrote: On 03/03/15 16:11, Simo Sorce wrote: On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. I do not like idea of auto upgrading during (re)start as well, it is
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 16:47, Martin Kosek wrote: On 03/03/2015 04:43 PM, Martin Basti wrote: On 03/03/15 16:11, Simo Sorce wrote: On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. I do not like idea of auto upgrading during (re)start as well, it is not expected by user. At least,
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 16:10, Martin Kosek wrote: On 03/03/2015 04:04 PM, Rob Crittenden wrote: Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) Only if there is a tty which means no asking during package update, which I thought was the idea. I just think it is rather unexpected to update a package during a restart. So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I'm beginning to have my own doubts about version, recognizing that there isn't exactly another obvious solution. Running the updates every time ipactl is run isn't great. The updates are not fast by any stretch, 29s on one VM, and we need to log whenever an update is done. My ipaupgrade log is 48M from 20 updates. How many times does one run ipactl restart when diagnosing a problem? My biggest concern with version is who keeps count and where? This is particularly problematic in packaged servers where changes are made without rebasing (Fedora and RHEL). Somewhere the version would need to be bumped with each release? Or only when updates are added? Or only when someone remembers? It just seems fragile and prone to human error unless you have some automatic version incrementor that takes this into consideration. If fallible version or slow updates are the only option then I'd have to go with slow updates if only to avoid a lot of support issues. And I really hate the idea of updates during service restart. rob Storing the version is something we just have to do, we need it for missing upgrade detection in FedUp case. We also need for container use case, to make sure that we can check whether we
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 3.3.2015 10:58, Martin Kosek wrote:
On 03/03/2015 09:36 AM, Petr Spacek wrote:
On 3.3.2015 09:33, Jan Cholasta wrote:
Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
On 03/03/15 07:31, Jan Cholasta wrote:
Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
On 02/03/15 13:12, Jan Cholasta wrote:
Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
On 03/02/2015 07:49 AM, Jan Cholasta wrote:
Hi,
Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
Hello all,
please read the design page, any objections/suggestions appreciated
http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
1)
* Merge server update commands into the one command
(ipa-server-upgrade)
So there is ipa-server-install to install the server,
ipa-server-install
--uninstall to uninstall the server and ipa-server-upgrade to
upgrade the
server. Maybe we should bring some consistency here and have one of:
a) ipa-server-install [--install], ipa-server-install
--uninstall,
ipa-server-install --upgrade
b) ipa-server-install [install], ipa-server-install uninstall,
ipa-server-install upgrade
c) ipa-server-install, ipa-server-uninstall,
ipa-server-upgrade
Long term, I think we want C. Besides other advantages, it will let
us have
independent sets of options, based on what you want to do.
2)
* Prevent to run IPA service, if code version and configuration
version does
not match
* ipactl should execute ipa-server-upgrade if needed
There should be no configuration version, configuration update
should be run
always. It's fast and hence does not need to be optimized like data
update by
using a monolithic version number, which brings more than a few
problems on its
own.
I do not agree in this section. Why would you like to run it always,
even if it
was fast? No run is still faster than fast run.
In the ideal case the installer would be idempotent and upgrade would
be re-running the installer and we should aim to do just that. We kind
of do that already, but there is a lot of code duplication in
installers and ipa-upgradeconfig (I would like to fix that when
refactoring installers). IMO it's better to always make 100% sure the
configuration is correct rather than to save a second or two.
I doesn't like this idea, if user wants to fix something, the one should
use --skip-version-check option, and the IPA upgrade will be executed.
Well, what I don't like is dealing with meaningless version numbers.
They are causing us grief in API versioning and I don't see why it
would be any different here.
However you must keep the version because of schema and data upgrade, so
why not to execute update as one batch instead of doing config upgrade
all the time, and then data upgrade only if required.
Because there is no exact mapping between version number and what features
are
actually available. A state file is tons better than a single version
number.
Some configuration upgrades, like adding new DNS related services,
requires new schema, how we can handle this?
This does not sound right. Could you be more specific?
Running schema upgrade every time?
What if a service changes in a way, the IPA configuration will not work?
Then it's a bug and needs to be fixed, like any other bug. IIRC there
was only one or two occurences of such bug in the past 3 years (I
remember sshd_config), so I don't think you have a strong case here.
Ok
The user will need to change it manually, but after each restart,
upgrade will change the value back into IPA required configuration which
will not work.
Says who? It's our code, we can do whatever we want, it doesn't have
to be dumb like this.
Yes, we have upgrade state file, but then the comparing of one value is
faster then checking each state if was executed.
How faster is that, like, few milliseconds? Are you seriously
considering this the right optimization in a process that is
magnitudes slower?
Ok the speed is not so important, but I still do not like the idea of
executing the code which is not needed to be executed, because I know
the version is the same as was before last restart, so nothing changed.
Weren't clever optimizations like this what got us into this whole
refactoring bussiness in the first place?
I very much agree with Honza. We should always start with something
stupidly-simply and enhance it later, when it is clear if it is really
necessary.
Do not over-engineer it from the very beginning.
I completely agree with starting stupid and simply and improving in time.
However, are we sure that what Honza proposed is the simple and stupid way?
Doing config upgrade only when needed and thus not depending on the efficiency
and idempotency of the config upgraders seems to me as *the* stupid and simple
way for upgrade refactoring.
Maybe I'm missing something but
if not state.get('dns_forward_zones_supported'):
# upgrade to forward zones here
seems less error-prone than
if version 4.0:
#
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 11:04 Petr Spacek napsal(a):
On 3.3.2015 10:58, Martin Kosek wrote:
On 03/03/2015 09:36 AM, Petr Spacek wrote:
On 3.3.2015 09:33, Jan Cholasta wrote:
Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
On 03/03/15 07:31, Jan Cholasta wrote:
Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
On 02/03/15 13:12, Jan Cholasta wrote:
Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
On 03/02/2015 07:49 AM, Jan Cholasta wrote:
Hi,
Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
Hello all,
please read the design page, any objections/suggestions appreciated
http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
1)
* Merge server update commands into the one command
(ipa-server-upgrade)
So there is ipa-server-install to install the server,
ipa-server-install
--uninstall to uninstall the server and ipa-server-upgrade to
upgrade the
server. Maybe we should bring some consistency here and have one of:
a) ipa-server-install [--install], ipa-server-install
--uninstall,
ipa-server-install --upgrade
b) ipa-server-install [install], ipa-server-install uninstall,
ipa-server-install upgrade
c) ipa-server-install, ipa-server-uninstall,
ipa-server-upgrade
Long term, I think we want C. Besides other advantages, it will let
us have
independent sets of options, based on what you want to do.
2)
* Prevent to run IPA service, if code version and configuration
version does
not match
* ipactl should execute ipa-server-upgrade if needed
There should be no configuration version, configuration update
should be run
always. It's fast and hence does not need to be optimized like data
update by
using a monolithic version number, which brings more than a few
problems on its
own.
I do not agree in this section. Why would you like to run it always,
even if it
was fast? No run is still faster than fast run.
In the ideal case the installer would be idempotent and upgrade would
be re-running the installer and we should aim to do just that. We kind
of do that already, but there is a lot of code duplication in
installers and ipa-upgradeconfig (I would like to fix that when
refactoring installers). IMO it's better to always make 100% sure the
configuration is correct rather than to save a second or two.
I doesn't like this idea, if user wants to fix something, the one should
use --skip-version-check option, and the IPA upgrade will be executed.
Well, what I don't like is dealing with meaningless version numbers.
They are causing us grief in API versioning and I don't see why it
would be any different here.
However you must keep the version because of schema and data upgrade, so
why not to execute update as one batch instead of doing config upgrade
all the time, and then data upgrade only if required.
Because there is no exact mapping between version number and what features are
actually available. A state file is tons better than a single version number.
Some configuration upgrades, like adding new DNS related services,
requires new schema, how we can handle this?
This does not sound right. Could you be more specific?
Running schema upgrade every time?
What if a service changes in a way, the IPA configuration will not work?
Then it's a bug and needs to be fixed, like any other bug. IIRC there
was only one or two occurences of such bug in the past 3 years (I
remember sshd_config), so I don't think you have a strong case here.
Ok
The user will need to change it manually, but after each restart,
upgrade will change the value back into IPA required configuration which
will not work.
Says who? It's our code, we can do whatever we want, it doesn't have
to be dumb like this.
Yes, we have upgrade state file, but then the comparing of one value is
faster then checking each state if was executed.
How faster is that, like, few milliseconds? Are you seriously
considering this the right optimization in a process that is
magnitudes slower?
Ok the speed is not so important, but I still do not like the idea of
executing the code which is not needed to be executed, because I know
the version is the same as was before last restart, so nothing changed.
Weren't clever optimizations like this what got us into this whole
refactoring bussiness in the first place?
I very much agree with Honza. We should always start with something
stupidly-simply and enhance it later, when it is clear if it is really
necessary.
Do not over-engineer it from the very beginning.
I completely agree with starting stupid and simply and improving in time.
However, are we sure that what Honza proposed is the simple and stupid way?
Doing config upgrade only when needed and thus not depending on the efficiency
and idempotency of the config upgraders seems to me as *the* stupid and simple
way for upgrade refactoring.
Maybe I'm missing something but
if not state.get('dns_forward_zones_supported'):
# upgrade to forward zones here
seems less error-prone than
if version 4.0:
# upgrade to forward
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 11:06 AM, Jan Cholasta wrote: Dne 3.3.2015 v 11:04 Petr Spacek napsal(a): On 3.3.2015 10:58, Martin Kosek wrote: On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Maybe I'm missing something but if not
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 11:04, Petr Spacek wrote:
On 3.3.2015 10:58, Martin Kosek wrote:
On 03/03/2015 09:36 AM, Petr Spacek wrote:
On 3.3.2015 09:33, Jan Cholasta wrote:
Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
On 03/03/15 07:31, Jan Cholasta wrote:
Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
On 02/03/15 13:12, Jan Cholasta wrote:
Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
On 03/02/2015 07:49 AM, Jan Cholasta wrote:
Hi,
Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
Hello all,
please read the design page, any objections/suggestions appreciated
http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
1)
* Merge server update commands into the one command
(ipa-server-upgrade)
So there is ipa-server-install to install the server,
ipa-server-install
--uninstall to uninstall the server and ipa-server-upgrade to
upgrade the
server. Maybe we should bring some consistency here and have one of:
a) ipa-server-install [--install], ipa-server-install
--uninstall,
ipa-server-install --upgrade
b) ipa-server-install [install], ipa-server-install uninstall,
ipa-server-install upgrade
c) ipa-server-install, ipa-server-uninstall,
ipa-server-upgrade
Long term, I think we want C. Besides other advantages, it will let
us have
independent sets of options, based on what you want to do.
2)
* Prevent to run IPA service, if code version and configuration
version does
not match
* ipactl should execute ipa-server-upgrade if needed
There should be no configuration version, configuration update
should be run
always. It's fast and hence does not need to be optimized like data
update by
using a monolithic version number, which brings more than a few
problems on its
own.
I do not agree in this section. Why would you like to run it always,
even if it
was fast? No run is still faster than fast run.
In the ideal case the installer would be idempotent and upgrade would
be re-running the installer and we should aim to do just that. We kind
of do that already, but there is a lot of code duplication in
installers and ipa-upgradeconfig (I would like to fix that when
refactoring installers). IMO it's better to always make 100% sure the
configuration is correct rather than to save a second or two.
I doesn't like this idea, if user wants to fix something, the one should
use --skip-version-check option, and the IPA upgrade will be executed.
Well, what I don't like is dealing with meaningless version numbers.
They are causing us grief in API versioning and I don't see why it
would be any different here.
However you must keep the version because of schema and data upgrade, so
why not to execute update as one batch instead of doing config upgrade
all the time, and then data upgrade only if required.
Because there is no exact mapping between version number and what features are
actually available. A state file is tons better than a single version number.
Some configuration upgrades, like adding new DNS related services,
requires new schema, how we can handle this?
This does not sound right. Could you be more specific?
Running schema upgrade every time?
What if a service changes in a way, the IPA configuration will not work?
Then it's a bug and needs to be fixed, like any other bug. IIRC there
was only one or two occurences of such bug in the past 3 years (I
remember sshd_config), so I don't think you have a strong case here.
Ok
The user will need to change it manually, but after each restart,
upgrade will change the value back into IPA required configuration which
will not work.
Says who? It's our code, we can do whatever we want, it doesn't have
to be dumb like this.
Yes, we have upgrade state file, but then the comparing of one value is
faster then checking each state if was executed.
How faster is that, like, few milliseconds? Are you seriously
considering this the right optimization in a process that is
magnitudes slower?
Ok the speed is not so important, but I still do not like the idea of
executing the code which is not needed to be executed, because I know
the version is the same as was before last restart, so nothing changed.
Weren't clever optimizations like this what got us into this whole
refactoring bussiness in the first place?
I very much agree with Honza. We should always start with something
stupidly-simply and enhance it later, when it is clear if it is really
necessary.
Do not over-engineer it from the very beginning.
I completely agree with starting stupid and simply and improving in time.
However, are we sure that what Honza proposed is the simple and stupid way?
Doing config upgrade only when needed and thus not depending on the efficiency
and idempotency of the config upgraders seems to me as *the* stupid and simple
way for upgrade refactoring.
Maybe I'm missing something but
if not state.get('dns_forward_zones_supported'):
# upgrade to forward zones here
seems less error-prone than
if version 4.0:
# upgrade to forward zones here
# make me
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 12:08 Martin Kosek napsal(a):
On 03/03/2015 11:06 AM, Jan Cholasta wrote:
Dne 3.3.2015 v 11:04 Petr Spacek napsal(a):
On 3.3.2015 10:58, Martin Kosek wrote:
On 03/03/2015 09:36 AM, Petr Spacek wrote:
On 3.3.2015 09:33, Jan Cholasta wrote:
Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
On 03/03/15 07:31, Jan Cholasta wrote:
Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
On 02/03/15 13:12, Jan Cholasta wrote:
Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
On 03/02/2015 07:49 AM, Jan Cholasta wrote:
Hi,
Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
Hello all,
please read the design page, any objections/suggestions appreciated
http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
1)
* Merge server update commands into the one command
(ipa-server-upgrade)
So there is ipa-server-install to install the server,
ipa-server-install
--uninstall to uninstall the server and ipa-server-upgrade to
upgrade the
server. Maybe we should bring some consistency here and have one of:
a) ipa-server-install [--install], ipa-server-install
--uninstall,
ipa-server-install --upgrade
b) ipa-server-install [install], ipa-server-install uninstall,
ipa-server-install upgrade
c) ipa-server-install, ipa-server-uninstall,
ipa-server-upgrade
Long term, I think we want C. Besides other advantages, it will let
us have
independent sets of options, based on what you want to do.
2)
* Prevent to run IPA service, if code version and configuration
version does
not match
* ipactl should execute ipa-server-upgrade if needed
There should be no configuration version, configuration update
should be run
always. It's fast and hence does not need to be optimized like data
update by
using a monolithic version number, which brings more than a few
problems on its
own.
I do not agree in this section. Why would you like to run it always,
even if it
was fast? No run is still faster than fast run.
In the ideal case the installer would be idempotent and upgrade would
be re-running the installer and we should aim to do just that. We kind
of do that already, but there is a lot of code duplication in
installers and ipa-upgradeconfig (I would like to fix that when
refactoring installers). IMO it's better to always make 100% sure the
configuration is correct rather than to save a second or two.
I doesn't like this idea, if user wants to fix something, the one should
use --skip-version-check option, and the IPA upgrade will be executed.
Well, what I don't like is dealing with meaningless version numbers.
They are causing us grief in API versioning and I don't see why it
would be any different here.
However you must keep the version because of schema and data upgrade, so
why not to execute update as one batch instead of doing config upgrade
all the time, and then data upgrade only if required.
Because there is no exact mapping between version number and what features
are
actually available. A state file is tons better than a single version number.
Some configuration upgrades, like adding new DNS related services,
requires new schema, how we can handle this?
This does not sound right. Could you be more specific?
Running schema upgrade every time?
What if a service changes in a way, the IPA configuration will not work?
Then it's a bug and needs to be fixed, like any other bug. IIRC there
was only one or two occurences of such bug in the past 3 years (I
remember sshd_config), so I don't think you have a strong case here.
Ok
The user will need to change it manually, but after each restart,
upgrade will change the value back into IPA required configuration which
will not work.
Says who? It's our code, we can do whatever we want, it doesn't have
to be dumb like this.
Yes, we have upgrade state file, but then the comparing of one value is
faster then checking each state if was executed.
How faster is that, like, few milliseconds? Are you seriously
considering this the right optimization in a process that is
magnitudes slower?
Ok the speed is not so important, but I still do not like the idea of
executing the code which is not needed to be executed, because I know
the version is the same as was before last restart, so nothing changed.
Weren't clever optimizations like this what got us into this whole
refactoring bussiness in the first place?
I very much agree with Honza. We should always start with something
stupidly-simply and enhance it later, when it is clear if it is really
necessary.
Do not over-engineer it from the very beginning.
I completely agree with starting stupid and simply and improving in time.
However, are we sure that what Honza proposed is the simple and stupid way?
Doing config upgrade only when needed and thus not depending on the efficiency
and idempotency of the config upgraders seems to me as *the* stupid and simple
way for upgrade refactoring.
Maybe I'm missing something but
if not state.get('dns_forward_zones_supported'):
# upgrade to forward
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. I have no more objections. * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 3.3.2015 11:01, Jan Cholasta wrote: I would very much prefer to do it the other way around, so that most bugs in the code are caught early, instead of hidden behind the version comparison. +1 -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do upgrade only if needed not every restart. If there is nothing to upgrade, nothing will be upgraded. The effect is the same. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? Sorry, I will add more details there, if this is not clear. ipa-upgrateconfig will be removed ipa-ldap-updater will
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. What if we will need some update which needs to execute time-consuming system check during every upgrade in future? User can always run the upgrade manually, with --skip-version-check, and then configuration plugins will decide if the upgrade is needed. My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 11:05 AM, Jan Cholasta wrote: Dne 3.3.2015 v 10:58 Martin Kosek napsal(a): On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. How exactly is adding another logic layer on top of the configuration update more simple and stupid? Maybe it isn't. I
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. Petr^2 Spacek My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... I want to do upgrade only if needed not every restart. If there is nothing to upgrade, nothing will be upgraded. The effect is the same. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 10:58 Martin Kosek napsal(a): On 03/03/2015 09:36 AM, Petr Spacek wrote: On 3.3.2015 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? I very much agree with Honza. We should always start with something stupidly-simply and enhance it later, when it is clear if it is really necessary. Do not over-engineer it from the very beginning. I completely agree with starting stupid and simply and improving in time. However, are we sure that what Honza proposed is the simple and stupid way? Doing config upgrade only when needed and thus not depending on the efficiency and idempotency of the config upgraders seems to me as *the* stupid and simple way for upgrade refactoring. How exactly is adding another logic layer on top of the configuration update more simple and stupid? Martin -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 11:00 Martin Basti napsal(a): On 03/03/15 10:55, Jan Cholasta wrote: Dne 3.3.2015 v 09:55 Martin Basti napsal(a): On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: ... 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Right, but what's your point? We are not discussing order of updates here, I'm perfectly fine with schema updates being done before configuration updates. So you want to run schema update before configuration upgrade every restart? OR you want to run schema update if needed based on version, and then run configuration upgrade? I don't really care, as long as the schema is up-to-date, but I guess there is no harm in always running schema update either. -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 09:55 AM, Martin Basti wrote: On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. What if we will need some update which needs to execute time-consuming system check during every upgrade in future? User can always run the upgrade manually, with --skip-version-check, and then configuration plugins will decide if the upgrade is needed. I tend to agree with Martin, I would prefer to be on the safe side and not run config upgrades every time, unless we are explicitly
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 3.3.2015 v 09:55 Martin Basti napsal(a): On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Right, but what's your point? We are not discussing order of updates here, I'm perfectly fine with schema updates being done before configuration updates. Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. Yes, then why do we need another one, especially so when it does not provide any significant speed-up? What if we will need some update which needs to execute time-consuming system check during every upgrade in future? Then we deal with the optimization in the future, instead of
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 2.3.2015 18:54, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. I have no more objections. IMHO the problem with long-running RPM upgrade should be approached from the other way: Just print message 'IPA server upgrade is running, press CTRL+C if you want to destroy your IPA server'. bind-dyndb-ldap prints a message about SELinux in RPM scriptlets for couple releases now and nobody complained (yet? :-). Petr^2 Spacek * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/15 10:55, Jan Cholasta wrote: Dne 3.3.2015 v 09:55 Martin Basti napsal(a): On 03/03/15 09:33, Jan Cholasta wrote: Dne 3.3.2015 v 09:06 Martin Basti napsal(a): On 03/03/15 07:31, Jan Cholasta wrote: Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. However you must keep the version because of schema and data upgrade, so why not to execute update as one batch instead of doing config upgrade all the time, and then data upgrade only if required. Because there is no exact mapping between version number and what features are actually available. A state file is tons better than a single version number. Some configuration upgrades, like adding new DNS related services, requires new schema, how we can handle this? This does not sound right. Could you be more specific? at least ipa-dnskeysyncd service, requires updated schema for keys metadata. This service is mandratory for DNS, so it is newly configured during upgrade. Now it works because schema update is the first, so during configuration upgrade we have actual schema. Right, but what's your point? We are not discussing order of updates here, I'm perfectly fine with schema updates being done before configuration updates. So you want to run schema update before configuration upgrade every restart? OR you want to run schema update if needed based on version, and then run configuration upgrade? Running schema upgrade every time? What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. Ok The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? Ok the speed is not so important, but I still do not like the idea of executing the code which is not needed to be executed, because I know the version is the same as was before last restart, so nothing changed. Weren't clever optimizations like this what got us into this whole refactoring bussiness in the first place? The clever optimizations worked in past, but IPA grown and now contains constraints/requirements which nobody expected. Yes, then why do we need another one, especially so when
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Tue, 2015-03-03 at 15:33 +0100, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? It's a hard problem. First of all DS should never deadlock, we shouldn't change all our process because one component has a bug If DS deadlocks, it will do so whether it is run in RPM or not, we need to add some way to terminate the upgrade ourselves if it lasts too long I guess. The problem with an asynchronous restart is ... when do you do that ? What triggers it ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On Tue, 2015-03-03 at 15:40 +0100, Martin Basti wrote: On 03/03/15 15:33, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? Martin The plan is do upgrade during RPM transaction if possible. If not then ipactl start, will show warning for user to do manual upgrade (Rob wanted it in this way, not doing auto upgrade by ipactl) So the fedup case is: RPM upgrade failed, ipactl start will detect version mismatch, show error and prompt user to run ipa-server-upgrade I think this is wrong, sorry. It means unattended installs will just fail to restart and require manual intervention. At the very least this process needs to be conditional, and upgrade needs to be done automatically. If the admin insist he can set something in default.conf to block autoupgrades or something. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/03/2015 03:49 PM, Simo Sorce wrote: On Tue, 2015-03-03 at 15:33 +0100, Martin Kosek wrote: On 03/03/2015 03:16 PM, Simo Sorce wrote: On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote: On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. There is no use case for using the DM password. +1, so we will only use LDAPI and ditch DM password options and querying that we now have with ipa-ldap-updater? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. This is true, stopping IPA and requiring manual intervention is not ok. What is the plan then? Keep upgrades done during RPM transaction? Note that RPM transaction was already stuck several times because IPA, or rather DS, deadlocked. We also need to address https://fedorahosted.org/freeipa/ticket/3849. The original plan was to do the upgrade during ipactl start, this would fix this ticket. Alternatively, should we remove the upgrade from RPM scriptlet and only call asynchronous systemctl restart ipa.service that would trigger the upgrade in separate process and log results in ipa.service? It's a hard problem. First of all DS should never deadlock, we shouldn't change all our process because one component has a bug If DS deadlocks, it will do so whether it is run in RPM or not, we need to add some way to terminate the upgrade ourselves if it lasts too long I guess. The problem with an asynchronous restart is ... when do you do that ? What triggers it ? systemctl restart ipa.service --no-block :-) If there is no better place for the upgrade outside of RPM transaction, I am fine, as long as ipactl writes the warning for FedUp users, as Martin mentioned, to make sure they do not run un-upgraded IPA. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 2.3.2015 v 13:51 Martin Basti napsal(a): On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. Well, what I don't like is dealing with meaningless version numbers. They are causing us grief in API versioning and I don't see why it would be any different here. What if a service changes in a way, the IPA configuration will not work? Then it's a bug and needs to be fixed, like any other bug. IIRC there was only one or two occurences of such bug in the past 3 years (I remember sshd_config), so I don't think you have a strong case here. The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Says who? It's our code, we can do whatever we want, it doesn't have to be dumb like this. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. How faster is that, like, few milliseconds? Are you seriously considering this the right optimization in a process that is magnitudes slower? My personal opinion is, application should not try to fix itself every restart. One could say that application should not try to upgrade itself every restart, but here we are, doing it anyway... In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? Sorry, I will add more details there, if this is not clear. ipa-upgrateconfig will be removed ipa-ldap-updater will not be able to do overall update, you will need to specify options and update file. 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) I have no strong feelings on this one, but IMO it is always better to have some plan B if we choose to indeed implement some yet unforeseen plugin based capability... I doubt that will happen, but if it does, we can always add plan-b-plugin directive. I do not insist on update-plugin, I just wanted to be more specific which type of plugin is expected there. Well, the names of the files end with .update and they are located in /usr/share/ipa/updates, I think that's enough hints as to what type of plugin is expected. 5) New class UpdatePlugin is used for all update plugins. Just reuse the existing Updater class, no need to reinvent the wheel. 6) I wonder why configuration update is done after data update and not before. I know it's been like that for a long time, but it
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 27/02/15 20:45, Rob Crittenden wrote: Martin Basti wrote: On 26/02/15 10:45, Petr Spacek wrote: On 25.2.2015 17:49, Martin Basti wrote: On 25/02/15 17:15, Petr Spacek wrote: On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. 4.1.3-2 4.1.3-12 in lexicographic order, this will not fit. Well, sure, but it allows you to use 00-a 01-b and renumber it to 001-a 002-b at will without changes to code. (Lexicographic order is what 'ls' uses by default so you can see the real ordering at any time very easily.) Also, as you pointed out, it allows you to do things like 12.345-a 12.666-bbb without changing code, again :-) Oh stupid me, I read it wrong, I replied with IPA version compare. sounds good to me, any objections anyone? This makes sense as long as we don't abuse it. The numbers are there to apply some amount of order but flexibility is good, and will avoid the problem of having humongous update files. So it is not clear to me, should we use 4 digit numbers, or just lexicographic order? I'm fine with allowing DM given that it allows running as non-root (pretty much the only condition that ldapi wouldn't work), but I think a full upgrade will fail w/o root given that you are combining the two commands. I'm not sure, if I get it. The ipa-server-upgrade has to be run under root user (ldapi, service upgrades). If LDAPI failed, user may use DM password to do LDAP update (this should fix LDAPI settings) but root user is still required to update services. So root user is always required to do this. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Looks really good. rob -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Martin Basti wrote: On 27/02/15 20:45, Rob Crittenden wrote: Martin Basti wrote: On 26/02/15 10:45, Petr Spacek wrote: On 25.2.2015 17:49, Martin Basti wrote: On 25/02/15 17:15, Petr Spacek wrote: On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. 4.1.3-2 4.1.3-12 in lexicographic order, this will not fit. Well, sure, but it allows you to use 00-a 01-b and renumber it to 001-a 002-b at will without changes to code. (Lexicographic order is what 'ls' uses by default so you can see the real ordering at any time very easily.) Also, as you pointed out, it allows you to do things like 12.345-a 12.666-bbb without changing code, again :-) Oh stupid me, I read it wrong, I replied with IPA version compare. sounds good to me, any objections anyone? This makes sense as long as we don't abuse it. The numbers are there to apply some amount of order but flexibility is good, and will avoid the problem of having humongous update files. So it is not clear to me, should we use 4 digit numbers, or just lexicographic order? Use lexicographic order with number prefixes so things look the way the always have but it will be more flexible going forward. I don't know that IPA will ever actually run out of enough number prefixes for this to be a serious problem but plan for the worst I guess. I'd just update the current README that recommends certain levels with information on the new sorting. The order within a given numeric subgroup is only important in a minority of cases. I'm fine with allowing DM given that it allows running as non-root (pretty much the only condition that ldapi wouldn't work), but I think a full upgrade will fail w/o root given that you are combining the two commands. I'm not sure, if I get it. The ipa-server-upgrade has to be run under root user (ldapi, service upgrades). If LDAPI failed, user may use DM password to do LDAP update (this should fix LDAPI settings) but root user is still required to update services. So root user is always required to do this. But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) I have no strong feelings on this one, but IMO it is always better to have some plan B if we choose to indeed implement some yet unforeseen plugin based capability... I doubt that will happen, but if it does, we can always add plan-b-plugin directive. 5) New class UpdatePlugin is used for all update plugins. Just reuse the existing Updater class, no need to reinvent the wheel. 6) I wonder why configuration update is done after data update and not before. I know it's been like that for a long time, but it seems kind of unnatural to me, especially now when schema update is separate from data update. (Rob?) 7) keep --test option and fix the plugins which do not respect the option Just a note, I believe this ticket is related: https://fedorahosted.org/freeipa/ticket/3448. Good work overall! Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 02/03/15 13:12, Jan Cholasta wrote: Dne 2.3.2015 v 12:23 Martin Kosek napsal(a): On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the ideal case the installer would be idempotent and upgrade would be re-running the installer and we should aim to do just that. We kind of do that already, but there is a lot of code duplication in installers and ipa-upgradeconfig (I would like to fix that when refactoring installers). IMO it's better to always make 100% sure the configuration is correct rather than to save a second or two. I doesn't like this idea, if user wants to fix something, the one should use --skip-version-check option, and the IPA upgrade will be executed. What if a service changes in a way, the IPA configuration will not work? The user will need to change it manually, but after each restart, upgrade will change the value back into IPA required configuration which will not work. Yes, we have upgrade state file, but then the comparing of one value is faster then checking each state if was executed. My personal opinion is, application should not try to fix itself every restart. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. Correct, but I was talking about configuration file updates, not (re)starts, which have to always be done in ipactl anyway. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? Sorry, I will add more details there, if this is not clear. ipa-upgrateconfig will be removed ipa-ldap-updater will not be able to do overall update, you will need to specify options and update file. 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) I have no strong feelings on this one, but IMO it is always better to have some plan B if we choose to indeed implement some yet unforeseen plugin based capability... I doubt that will happen, but if it does, we can always add plan-b-plugin directive. I do not insist on update-plugin, I just wanted to be more specific which type of plugin is expected there. 5) New class UpdatePlugin is used for all update plugins. Just reuse the existing Updater class, no need to reinvent the wheel. 6) I wonder why configuration update is done after data update and not before. I know it's been like that for a long time, but it seems kind of unnatural to me, especially now when schema update is separate from data update. (Rob?) We need schema update first, but I haven't found any services which need to have updated data (I might be wrong) 7) keep --test option and fix the plugins which do not respect the option Just a note, I believe this ticket is related: https://fedorahosted.org/freeipa/ticket/3448. Good work overall! Honza -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/02/2015 07:49 AM, Jan Cholasta wrote: Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade Long term, I think we want C. Besides other advantages, it will let us have independent sets of options, based on what you want to do. 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. I do not agree in this section. Why would you like to run it always, even if it was fast? No run is still faster than fast run. In the past, I do not recall ipa-upgradeconfig as being really fast, especially certmonger/Dogtag related updates were really slow thank to service restarts, etc. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) I have no strong feelings on this one, but IMO it is always better to have some plan B if we choose to indeed implement some yet unforeseen plugin based capability... 5) New class UpdatePlugin is used for all update plugins. Just reuse the existing Updater class, no need to reinvent the wheel. 6) I wonder why configuration update is done after data update and not before. I know it's been like that for a long time, but it seems kind of unnatural to me, especially now when schema update is separate from data update. (Rob?) 7) keep --test option and fix the plugins which do not respect the option Just a note, I believe this ticket is related: https://fedorahosted.org/freeipa/ticket/3448. Good work overall! Honza ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: On 27/02/15 20:45, Rob Crittenden wrote: Martin Basti wrote: On 26/02/15 10:45, Petr Spacek wrote: On 25.2.2015 17:49, Martin Basti wrote: On 25/02/15 17:15, Petr Spacek wrote: On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. 4.1.3-2 4.1.3-12 in lexicographic order, this will not fit. Well, sure, but it allows you to use 00-a 01-b and renumber it to 001-a 002-b at will without changes to code. (Lexicographic order is what 'ls' uses by default so you can see the real ordering at any time very easily.) Also, as you pointed out, it allows you to do things like 12.345-a 12.666-bbb without changing code, again :-) Oh stupid me, I read it wrong, I replied with IPA version compare. sounds good to me, any objections anyone? This makes sense as long as we don't abuse it. The numbers are there to apply some amount of order but flexibility is good, and will avoid the problem of having humongous update files. So it is not clear to me, should we use 4 digit numbers, or just lexicographic order? Use lexicographic order with number prefixes so things look the way the always have but it will be more flexible going forward. I don't know that IPA will ever actually run out of enough number prefixes for this to be a serious problem but plan for the worst I guess. I'd just update the current README that recommends certain levels with information on the new sorting. The order within a given numeric subgroup is only important in a minority of cases. Ok I'm fine with allowing DM given that it allows running as non-root (pretty much the only condition that ldapi wouldn't work), but I think a full upgrade will fail w/o root given that you are combining the two commands. I'm not sure, if I get it. The ipa-server-upgrade has to be run under root user (ldapi, service upgrades). If LDAPI failed, user may use DM password to do LDAP update (this should fix LDAPI settings) but root user is still required to update services. So root user is always required to do this. But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 02/03/15 18:28, Martin Kosek wrote: On 03/02/2015 06:12 PM, Martin Basti wrote: On 02/03/15 15:43, Rob Crittenden wrote: Martin Basti wrote: ... But you haven't explained any case why LDAPI would fail. If LDAPI fails then you've got more serious problems that I'm not sure binding as DM is going to solve. The only case where DM would be handy IMHO is either some worst case scenario upgrade where 389-ds is up but not binding to LDAPI or if you want to allow running LDAP updates as non-root. I don't know cases when LDAPI would failed, except the case LDAPI is misconfigured by user, or disabled by user. Wasn't LDAPI needed for the DM password less upgrade so that upgrader could simply bind as root with EXTERNAL auth? We can do upgrade in both way, using LDAPI or using DM password, preferred is LDAPI. Question is, what is the use case for using DM password instead of LDAPI during upgrade. It is not big effort to keep both DM binding and LDAPI in code. A user can always found som unexpected use case for LDAP update with DM password. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. In non-container enviroment, we can still use upgrade during RPM transaction. So you suggest not to do update automaticaly, just write Error the IPA upgrade is required? People do all sorts of strange things. Installing the packages with --no-script isn't in the range of impossible. A prompt, and I'm not saying it's a great idea, is 2 lines of code. I guess it just makes me nervous. So lets summarize this: * DO upgrade if possible during RPM transaction Umm, I thought we want to get rid of running upgrade during RPM transaction. It is extremely difficult to debug upgrade stuck during RPM transaction, it also makes RPM upgrade run longer than needed. It also makes admins nervous when their rpm upgrade is suddenly waiting right before the end. I even see the fingers slowly reaching to CTRL+C combo... (You can see the consequences) People are used to have IPA upgraded and ready after RPM upgrade. They may be shocked if IPA services will be in shutdown state after RPM transaction. I have no more objections. * ipactl will NOT run upgrade, just print Error: 'please upgrade ' * User has to run ipa-server-upgrade manually Does I understand it correctly? With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? For this big warning everywhere. The main problem is try to run older IPA with newer data. In containers the problem is to run different platform specific versions which differ in functionality/bugfixes etc.. Ok, pretty much the things I was thinking as well. A scary warning definitely seems warranted. Where does platform come from? I'm wondering how Debian will handle this. platform is derived from ipaplatform file which is used with the particular build. Debian should have own platform file. Ok, I'd add that detail to the design. rob -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Hi, Dne 24.2.2015 v 19:10 Martin Basti napsal(a): Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring 1) * Merge server update commands into the one command (ipa-server-upgrade) So there is ipa-server-install to install the server, ipa-server-install --uninstall to uninstall the server and ipa-server-upgrade to upgrade the server. Maybe we should bring some consistency here and have one of: a) ipa-server-install [--install], ipa-server-install --uninstall, ipa-server-install --upgrade b) ipa-server-install [install], ipa-server-install uninstall, ipa-server-install upgrade c) ipa-server-install, ipa-server-uninstall, ipa-server-upgrade 2) * Prevent to run IPA service, if code version and configuration version does not match * ipactl should execute ipa-server-upgrade if needed There should be no configuration version, configuration update should be run always. It's fast and hence does not need to be optimized like data update by using a monolithic version number, which brings more than a few problems on its own. 3) * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater Even without arguments? Is ipactl now the only right place to trigger manual update? 4) Plugins are called from update files, using new directive update-plugin:plugin-name Why update-plugin and not just plugin? Do you expect other kinds of plugins to be called from update files in the future? (I certainly don't.) 5) New class UpdatePlugin is used for all update plugins. Just reuse the existing Updater class, no need to reinvent the wheel. 6) I wonder why configuration update is done after data update and not before. I know it's been like that for a long time, but it seems kind of unnatural to me, especially now when schema update is separate from data update. (Rob?) 7) keep --test option and fix the plugins which do not respect the option Just a note, I believe this ticket is related: https://fedorahosted.org/freeipa/ticket/3448. Good work overall! Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
Martin Basti wrote: On 26/02/15 10:45, Petr Spacek wrote: On 25.2.2015 17:49, Martin Basti wrote: On 25/02/15 17:15, Petr Spacek wrote: On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. 4.1.3-2 4.1.3-12 in lexicographic order, this will not fit. Well, sure, but it allows you to use 00-a 01-b and renumber it to 001-a 002-b at will without changes to code. (Lexicographic order is what 'ls' uses by default so you can see the real ordering at any time very easily.) Also, as you pointed out, it allows you to do things like 12.345-a 12.666-bbb without changing code, again :-) Oh stupid me, I read it wrong, I replied with IPA version compare. sounds good to me, any objections anyone? This makes sense as long as we don't abuse it. The numbers are there to apply some amount of order but flexibility is good, and will avoid the problem of having humongous update files. I'm fine with allowing DM given that it allows running as non-root (pretty much the only condition that ldapi wouldn't work), but I think a full upgrade will fail w/o root given that you are combining the two commands. On ipactl, would it be overkill if there is a tty to prompt the user to upgrade? In a non-container world it might be surprising to have an upgrade happen esp since upgrades take a while. With --skip-version-check what sorts of problems can we foresee? I assume a big warning will be added to at least the man page, if not the cli? Where does platform come from? I'm wondering how Debian will handle this. Looks really good. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 26/02/15 10:45, Petr Spacek wrote: On 25.2.2015 17:49, Martin Basti wrote: On 25/02/15 17:15, Petr Spacek wrote: On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. 4.1.3-2 4.1.3-12 in lexicographic order, this will not fit. Well, sure, but it allows you to use 00-a 01-b and renumber it to 001-a 002-b at will without changes to code. (Lexicographic order is what 'ls' uses by default so you can see the real ordering at any time very easily.) Also, as you pointed out, it allows you to do things like 12.345-a 12.666-bbb without changing code, again :-) Oh stupid me, I read it wrong, I replied with IPA version compare. sounds good to me, any objections anyone? Petr^2 Spacek To resolve issues mentioned above only one command should do upgrade: ipa-server-upgrade. I very much agree with this. ipa-server-upgrade characteristics ... 4. LDAP data update (+ update plugins) 5. upgrade configuration At this point I would appreciate explanatory text what is 'LDAP data update' and what is 'upgrade configuration'. Maybe some examples could be enough. LDAP data update == upgrading data stored in LDAP (user data + cn=config) upgrade configuration == upgrading configuration of services in filesystem (apache, named) I will add some explanation there. ipactl checks if installed version and version stored in LDAP are the same: ... ipactl start|restart option --force overrides this check. I would like to see a separate option. --force currently skips rollback if some services could not start but this is fundamentally different from version/upgrade checks. Ohh, good catch thank you, maybe --skip-version-check ? Sounds fine to me. ipa-server-upgrade --version show program's version number and exit Maybe it could print code version + data version (if available). It could be handy debugging tool. Good idea thanks ipa-server-upgrade --test Note: for developing only Is it really worth the effort to keep the option and invest more time in it? I do not expect any extra effort (except fixing 3 plugins - 6 lines of code approx), so if it will help to develop updates it could stay there (personally I do not use it, usually updates broke during write to server on some constraints) Okay, I thought that it is broken significantly. -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 25.2.2015 17:49, Martin Basti wrote: On 25/02/15 17:15, Petr Spacek wrote: On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. 4.1.3-2 4.1.3-12 in lexicographic order, this will not fit. Well, sure, but it allows you to use 00-a 01-b and renumber it to 001-a 002-b at will without changes to code. (Lexicographic order is what 'ls' uses by default so you can see the real ordering at any time very easily.) Also, as you pointed out, it allows you to do things like 12.345-a 12.666-bbb without changing code, again :-) Petr^2 Spacek To resolve issues mentioned above only one command should do upgrade: ipa-server-upgrade. I very much agree with this. ipa-server-upgrade characteristics ... 4. LDAP data update (+ update plugins) 5. upgrade configuration At this point I would appreciate explanatory text what is 'LDAP data update' and what is 'upgrade configuration'. Maybe some examples could be enough. LDAP data update == upgrading data stored in LDAP (user data + cn=config) upgrade configuration == upgrading configuration of services in filesystem (apache, named) I will add some explanation there. ipactl checks if installed version and version stored in LDAP are the same: ... ipactl start|restart option --force overrides this check. I would like to see a separate option. --force currently skips rollback if some services could not start but this is fundamentally different from version/upgrade checks. Ohh, good catch thank you, maybe --skip-version-check ? Sounds fine to me. ipa-server-upgrade --version show program's version number and exit Maybe it could print code version + data version (if available). It could be handy debugging tool. Good idea thanks ipa-server-upgrade --test Note: for developing only Is it really worth the effort to keep the option and invest more time in it? I do not expect any extra effort (except fixing 3 plugins - 6 lines of code approx), so if it will help to develop updates it could stay there (personally I do not use it, usually updates broke during write to server on some constraints) Okay, I thought that it is broken significantly. -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] IPA Server upgrade 4.2 design
On 24.2.2015 19:10, Martin Basti wrote: Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring Thank you for the design, I have only few nitpicks. Increase update files numbers range Update files number will be extended into 4 digits values. IMHO the dependency on particular number format should be removed altogether. It should be perfectly enough to say that updates are executed in ASCII lexicographic order and be done with it. To resolve issues mentioned above only one command should do upgrade: ipa-server-upgrade. I very much agree with this. ipa-server-upgrade characteristics ... 4. LDAP data update (+ update plugins) 5. upgrade configuration At this point I would appreciate explanatory text what is 'LDAP data update' and what is 'upgrade configuration'. Maybe some examples could be enough. ipactl checks if installed version and version stored in LDAP are the same: ... ipactl start|restart option --force overrides this check. I would like to see a separate option. --force currently skips rollback if some services could not start but this is fundamentally different from version/upgrade checks. ipa-server-upgrade --version show program's version number and exit Maybe it could print code version + data version (if available). It could be handy debugging tool. ipa-server-upgrade --testNote: for developing only Is it really worth the effort to keep the option and invest more time in it? -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] IPA Server upgrade 4.2 design
Hello all, please read the design page, any objections/suggestions appreciated http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
