Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
On 15.1.2013 16:44, Martin Kosek wrote: On 01/15/2013 04:17 PM, Rob Crittenden wrote: Martin Kosek wrote: On 01/15/2013 03:44 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 15:37 +0100, Martin Kosek wrote: On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Oh I guess I wasn't clear, I did not mean to try to restart the service immediately or multiple times, I meant to make sure that if the service was running when the *whole* update started to make sure it is still running when the whole update finishes. The scenario is: 1. ipa runnig 2. do upgrade 3. restart fails for some reason 4. update completes now what I would like to make sure is that if the restart failed at 3 we try a restart after 4 so that we try to get things up when all the updates are done. Makes sense ? Sort of. To be able to do this, I think we would need to at first get a list of all running services (as user may have purposefully shut down some service), then run the upgrades and check that all services in this list are still running at the end of the upgrade. If not, try to amend it. While this looks useful-ish, I would rather keep the patch 350 simple as we are close to the release and I do not want to get too wild. Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. Yes we should do a conditional restart only, and it is ok to proceeded if it fails, we want to complete the upgrade process in any case, not break out in the middle if at all possible. Simo. Right, I will send an updated patch which restarts the named/pki-ca service only if it is running. ACK on this patch as-is. I think we have room for improvement/discussion. Can you open a RFE ticket to investigate any further work we might want to do? Sure, this is the ticket: https://fedorahosted.org/freeipa/ticket/3351 Anyway, I rebased the patch also for master and ipa-3-1 and pushed it to all three branches, i.e. master, ipa-3-1, ipa-3-0. BTW bind-dyndb-ldap has a open ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/100 for handling KDC unavailability. It should be coordinated with IPA's bug triage. -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
When either dirsrv or krb5kdc is down, named service restart in
ipa-upgradeconfig will fail and cause a crash of the whole upgrade
process.
Rather only report a failure to restart the service and continue
with the upgrade as it does not need the named service running. Do
the same precaution for pki-ca service restart.
https://fedorahosted.org/freeipa/ticket/3350
From b86eb367bca2ddbe06568024e7c1bf5eed543ad5 Mon Sep 17 00:00:00 2001
From: Martin Kosek mko...@redhat.com
Date: Tue, 15 Jan 2013 13:17:53 +0100
Subject: [PATCH] Upgrade process should not crash on named restart
When either dirsrv or krb5kdc is down, named service restart in
ipa-upgradeconfig will fail and cause a crash of the whole upgrade
process.
Rather only report a failure to restart the service and continue
with the upgrade as it does not need the named service running. Do
the same precaution for pki-ca service restart.
https://fedorahosted.org/freeipa/ticket/3350
---
install/tools/ipa-upgradeconfig | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/install/tools/ipa-upgradeconfig b/install/tools/ipa-upgradeconfig
index 32a258b1652309f02dc8b756d758243bc35c8db8..900985d80113330120ab79490272a11cc862917c 100644
--- a/install/tools/ipa-upgradeconfig
+++ b/install/tools/ipa-upgradeconfig
@@ -661,12 +661,19 @@ def main():
if changed_psearch or changed_autoincrement:
# configuration has changed, restart the name server
root_logger.info('Changes to named.conf have been made, restart named')
-bindinstance.BindInstance(fstore).restart()
+bind = bindinstance.BindInstance(fstore)
+try:
+bind.restart()
+except ipautil.CalledProcessError, e:
+root_logger.error(Failed to restart %s: %s, bind.service_name, e)
ca_restart = ca_restart or enable_certificate_renewal(ca) or upgrade_ipa_profile(ca)
if ca_restart:
root_logger.info('pki-ca configuration changed, restart pki-ca')
-ca.restart(dogtag.configured_constants().PKI_INSTANCE_NAME)
+try:
+ca.restart(dogtag.configured_constants().PKI_INSTANCE_NAME)
+except ipautil.CalledProcessError, e:
+root_logger.error(Failed to restart %s: %s, ca.service_name, e)
if __name__ == '__main__':
installutils.run_script(main, operation_name='ipa-upgradeconfig')
--
1.7.11.7
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
Martin Kosek wrote: On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. That makes sense to me. I also wonder if, since we know about the dirsrv dependency, we shouldn't check that at the same time. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
On Tue, 2013-01-15 at 15:37 +0100, Martin Kosek wrote: On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Oh I guess I wasn't clear, I did not mean to try to restart the service immediately or multiple times, I meant to make sure that if the service was running when the *whole* update started to make sure it is still running when the whole update finishes. The scenario is: 1. ipa runnig 2. do upgrade 3. restart fails for some reason 4. update completes now what I would like to make sure is that if the restart failed at 3 we try a restart after 4 so that we try to get things up when all the updates are done. Makes sense ? Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. Yes we should do a conditional restart only, and it is ok to proceeded if it fails, we want to complete the upgrade process in any case, not break out in the middle if at all possible. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
On 01/15/2013 03:44 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 15:37 +0100, Martin Kosek wrote: On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Oh I guess I wasn't clear, I did not mean to try to restart the service immediately or multiple times, I meant to make sure that if the service was running when the *whole* update started to make sure it is still running when the whole update finishes. The scenario is: 1. ipa runnig 2. do upgrade 3. restart fails for some reason 4. update completes now what I would like to make sure is that if the restart failed at 3 we try a restart after 4 so that we try to get things up when all the updates are done. Makes sense ? Sort of. To be able to do this, I think we would need to at first get a list of all running services (as user may have purposefully shut down some service), then run the upgrades and check that all services in this list are still running at the end of the upgrade. If not, try to amend it. While this looks useful-ish, I would rather keep the patch 350 simple as we are close to the release and I do not want to get too wild. Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. Yes we should do a conditional restart only, and it is ok to proceeded if it fails, we want to complete the upgrade process in any case, not break out in the middle if at all possible. Simo. Right, I will send an updated patch which restarts the named/pki-ca service only if it is running. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
Martin Kosek wrote: On 01/15/2013 03:44 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 15:37 +0100, Martin Kosek wrote: On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Oh I guess I wasn't clear, I did not mean to try to restart the service immediately or multiple times, I meant to make sure that if the service was running when the *whole* update started to make sure it is still running when the whole update finishes. The scenario is: 1. ipa runnig 2. do upgrade 3. restart fails for some reason 4. update completes now what I would like to make sure is that if the restart failed at 3 we try a restart after 4 so that we try to get things up when all the updates are done. Makes sense ? Sort of. To be able to do this, I think we would need to at first get a list of all running services (as user may have purposefully shut down some service), then run the upgrades and check that all services in this list are still running at the end of the upgrade. If not, try to amend it. While this looks useful-ish, I would rather keep the patch 350 simple as we are close to the release and I do not want to get too wild. Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. Yes we should do a conditional restart only, and it is ok to proceeded if it fails, we want to complete the upgrade process in any case, not break out in the middle if at all possible. Simo. Right, I will send an updated patch which restarts the named/pki-ca service only if it is running. ACK on this patch as-is. I think we have room for improvement/discussion. Can you open a RFE ticket to investigate any further work we might want to do? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 350 Upgrade process should not crash on named restart
On 01/15/2013 04:17 PM, Rob Crittenden wrote: Martin Kosek wrote: On 01/15/2013 03:44 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 15:37 +0100, Martin Kosek wrote: On 01/15/2013 02:43 PM, Simo Sorce wrote: On Tue, 2013-01-15 at 13:30 +0100, Martin Kosek wrote: When either dirsrv or krb5kdc is down, named service restart in ipa-upgradeconfig will fail and cause a crash of the whole upgrade process. Rather only report a failure to restart the service and continue with the upgrade as it does not need the named service running. Do the same precaution for pki-ca service restart. https://fedorahosted.org/freeipa/ticket/3350 Shouldn't we note it failed and retry later ? Is there a risk it will be down at the end of the upgrade process ? Simo. Seems like an overkill to me. It would not certainly help in this case, because the processes that named requires are down. As Rob suggested, user upgrading the IPA may be running in a lower run level for example, it that case I think we may not even try to restart the service. Oh I guess I wasn't clear, I did not mean to try to restart the service immediately or multiple times, I meant to make sure that if the service was running when the *whole* update started to make sure it is still running when the whole update finishes. The scenario is: 1. ipa runnig 2. do upgrade 3. restart fails for some reason 4. update completes now what I would like to make sure is that if the restart failed at 3 we try a restart after 4 so that we try to get things up when all the updates are done. Makes sense ? Sort of. To be able to do this, I think we would need to at first get a list of all running services (as user may have purposefully shut down some service), then run the upgrades and check that all services in this list are still running at the end of the upgrade. If not, try to amend it. While this looks useful-ish, I would rather keep the patch 350 simple as we are close to the release and I do not want to get too wild. Now when I am thinking about it, maybe we should only try to restart if the service is running - because otherwise it would be started later and the changes that were done in scope of upgrade script would be applied. Yes we should do a conditional restart only, and it is ok to proceeded if it fails, we want to complete the upgrade process in any case, not break out in the middle if at all possible. Simo. Right, I will send an updated patch which restarts the named/pki-ca service only if it is running. ACK on this patch as-is. I think we have room for improvement/discussion. Can you open a RFE ticket to investigate any further work we might want to do? Sure, this is the ticket: https://fedorahosted.org/freeipa/ticket/3351 Anyway, I rebased the patch also for master and ipa-3-1 and pushed it to all three branches, i.e. master, ipa-3-1, ipa-3-0. Martin rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
