On Wed, 2012-09-12 at 18:43 +0200, Petr Viktorin wrote: > On 09/11/2012 09:38 PM, Rob Crittenden wrote: > > Rob Crittenden wrote: > >> Rob Crittenden wrote: > >>> Petr Viktorin wrote: > >>>> On 09/11/2012 04:38 PM, Rob Crittenden wrote: > >>>>> Ade Lee wrote: > >>>>>> On Tue, 2012-09-11 at 08:59 -0400, Rob Crittenden wrote: > >>>>>>> Petr Viktorin wrote: > >>>>>>>> On 09/11/2012 04:04 AM, Ade Lee wrote: > >>>>>>>>> On Mon, 2012-09-10 at 16:58 -0400, Rob Crittenden wrote: > >>>>>>>>>> Petr Viktorin wrote: > >>>>>>>>>>> Attaching rebased and squashed patches. I've done some testing > >>>>>>>>>>> with > >>>>>>>>>>> them > >>>>>>>>>>> but please test some more. > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Most of these aren't IPA issues, but dogtag issues. I'll try to > >>>>>>>>>> split > >>>>>>>>>> them out. > >>>>>>>>>> > >>>>>>>>>> IPA: > >>>>>>>>>> > >>>>>>>>>> For the configuration files in install/conf to be updated at rpm > >>>>>>>>>> update > >>>>>>>>>> time the VERSION needs to be incremented. > >>>>>>>> > >>>>>>>> These files should stay the same since on upgrade we're still > >>>>>>>> using a > >>>>>>>> Dogtag 9 style instance. The Dogtag 10 ports are only used in new > >>>>>>>> installs. > >>>>>>>> > >>>>>>>>>> The ipa package lacks any updated dogtag dependencies, so I > >>>>>>>>>> abused > >>>>>>>>>> it. > >>>>>>>> > >>>>>>>> What should the updated dependencies be? Since it should work with > >>>>>>>> both > >>>>>>>> dogtag 9 and 10, I don't see how they should change. > >>>>>>> > >>>>>>> I don't know either, but we need to prevent people from installing > >>>>>>> incompatible package combinations. > >>>>>>> > >>>>>> Would'nt the Conflicts: ipa < 3.0 in pki-ca mentioned below satisfy > >>>>>> this > >>>>>> requirement? The main concern is that you must have ipa 3.0 if you > >>>>>> have > >>>>>> dogtag 10. > >>>>>> > >>>>>> Given that dogtag is consumed by IPA though, it makes more sense to > >>>>>> put > >>>>>> the relevant conflicts in IPA rather than in dogtag. So in this > >>>>>> case, > >>>>>> that would mean putting Conflicts: pki-ca >= 10.0 in IPA 2.x. > >>>>>> Recall that dogtag 10 will only be officially available in f18+. > >>>>> > >>>>> That isn't enough. If a F-17 user with IPA 2.2 installed upgrades to > >>>>> F-18 they would be able to install dogtag 10 and blow up their IPA > >>>>> server. > >>>>> > >>>>>> > >>>>>>>>>> I installed IPA with dogtag 9 and created a replica. > >>>>>>>>>> > >>>>>>>>>> I updated the IPA bits, that worked fine. > >>>>>>>>>> > >>>>>>>>>> I updated to dogtag 10 and now the CA doesn't work on the master, > >>>>>>>>>> including starting the dogtag instance. Note that the rpm update > >>>>>>>>>> process > >>>>>>>>>> worked, no notice that the CA service didn't restart. > >>>>>>>>>> > >>>>>>>>> Did you try to restart the CA with selinux in permissive mode? > >>>>>>>>> This is > >>>>>>>>> still required right now until I get the selinux policy all > >>>>>>>>> straightened > >>>>>>>>> out. > >>>>>>>>> > >>>>>>>>> There is also a separate dogtag ticket (which is currently being > >>>>>>>>> worked > >>>>>>>>> on) to restart existing dogtag instances when dogtag is upgraded > >>>>>>>>> from > >>>>>>>>> 9->10. > >>>>>>>> > >>>>>>>> In permissive mode, this upgrade works for me. > >>>>>>> > >>>>>>> I was in enforcing mode but saw no AVCs. What is the ETA on fixing > >>>>>>> this? > >>>>>>> > >>>>>> > >>>>>> Within the next week or two, I need to finish the IPA merge database > >>>>>> patch first, and then co-ordinate changes with the selinux guys. > >>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Sometimes I do get this error intermittently: > >>>>>>>> > >>>>>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to > >>>>>>>> communicate with CMS (Service Temporarily Unavailable) > >>>>>>>> > >>>>>>>> Usually, waiting a couple of minutes clears this up. Perhaps we are > >>>>>>>> not > >>>>>>>> doing startup detection correctly. Ade mentioned that waiting for > >>>>>>>> ports > >>>>>>>> may not be ideal. How do we know if Dogtag is initialized? > >>>>>>> > >>>>>>> Years ago we had discussed with Andrew and Matt creating a URI that > >>>>>>> can > >>>>>>> be queried to determine dogtag status. I don't think that ever went > >>>>>>> anywhere. > >>>>>>> > >>>>>> Petr, this happens only on reboot, right? And not on regular > >>>>>> "service > >>>>>> ipa restart"? > >>>> > >>>> I've now seen it happen right after a 9 → 10 upgrade. > >>>> > >>>>>> Yeah, I remember this conversation - and even created a bug for it at > >>>>>> some point. This went away because the mechanism you were using > >>>>>> seemed > >>>>>> to be working. The timing may be very different now with tomcat 7 > >>>>>> and > >>>>>> systemd. I'll open a dogtag trac ticket for this. > >>>>> > >>>>> Ok. > >>>>> > >>>>>> > >>>>>>>> > >>>>>>>>>> Uninstalling failed because it tried to run pkidestroy and not > >>>>>>>>>> pkiremove. > >>>>>>>> > >>>>>>>> I was under the impression that pkidestroy was the correct > >>>>>>>> command to > >>>>>>>> remove an upgraded instance. I'll check with Ade. > >>>>>>>> > >>>>>>>>> I'll test this too. > >>>>>>>>> > >>>>>>>>>> The contents of the file passed to pkispawn should be logged > >>>>>>>>>> so we > >>>>>>>>>> can > >>>>>>>>>> see exactly what was passed in. > >>>>>>>>>> > >>>>>>>>> Its a pretty big file. You might want to only log the > >>>>>>>>> modifications. > >>>>>>>>> Or save the file somewhere. > >>>>>>>> > >>>>>>>> Our logs are pretty verbose, so that shouldn't be a problem. I'll > >>>>>>>> put it > >>>>>>>> in the next version of the patch. > >>>>>>> > >>>>>>> The question to ask is: would you need the contents of this file if > >>>>>>> all > >>>>>>> you got were logs and needed to evaluate why installation failed? In > >>>>>>> most cases this is yes. > >>>>>>> > >>>>>> Up to you guys. There is a patch I am working on in which I will be > >>>>>> logging the object that is being passed to the server from pkispawn. > >>>>>> That - and the diffs to the standard config file as I mentioned > >>>>>> above - > >>>>>> will likely be sufficient to debug almost all cases. > >>>>>> > >>>>>> Make sure not to log any passwords. > >>>>>> > >>>> > >>>> Thanks for the catch. Attaching updated patch that sanitizes the > >>>> passwords. > >>>> > >>>>>>>>>> DOGTAG: > >>>>>>>>>> > >>>>>>>>>> When upgrading using the dogtag-devel repo I had to specify > >>>>>>>>>> pki-tools.x86_64 otherwise it tried to install both 32 and 64-bit > >>>>>>>>>> versions (and failed). > >>>>>>>>>> > >>>>>>>>>> I ended up running: yum update pki-ca tomcatjss pki-tools.x86_64 > >>>>>>>>>> --enablerepo=dogtag-devel --enablerepo=updates-testing > >>>>>>>>>> > >>>>>>>>> We'll look into this. I think I know why this happens. > >>>>>>>>> > >>>>>>>>>> What happens if someone manually upgrades pki-ca without first > >>>>>>>>>> updating > >>>>>>>>>> ipa? I think that pki-ca is going to need a Conflicts ipa < > >>>>>>>>>> 3.0 in > >>>>>>>>>> it. > >>>>>>>>> > >>>>>>>>> We can add that. > >>>>>>>>> > >>>>>>>>>> certificate renewal failed. I spent far too long trying to figure > >>>>>>>>>> out > >>>>>>>>>> why tomcat wasn't listening on port 9180 but failed. I think > >>>>>>>>>> 9180 is > >>>>>>>>>> actually the old server, right? So another missing dependency > >>>>>>>>>> on a > >>>>>>>>>> fixed > >>>>>>>>>> certmonger? > >>>>>>>>>> > >>>>>>>>>> The best I could find was the certmonger error: > >>>>>>>>>> > >>>>>>>>>> ca-error: Error 7 connecting to > >>>>>>>>>> http://edsel.example.com:9180/ca/ee/ca/profileSubmit: Couldn't > >>>>>>>>>> connect > >>>>>>>>>> to server. > >>>>>>>>>> > >>>>>>>> > >>>>>>>> I'll set the certmonger min version to 0.60 as per Nalin's mail. > >>>>>>> > >>>>>>> Ok. > >>>>>>> > >>>>>>>>> Is this cert renewal on a dogtag 10 instance? Or the upgraded > >>>>>>>>> dogtag 9? > >>>>>>>>> If its dogtag 10, perhaps you do not have the certmonger version > >>>>>>>>> that > >>>>>>>>> has the relevant fix? If its dogtag 9, then we need to figure out > >>>>>>>>> whats > >>>>>>>>> going on. That reminds me - I need to file a bug to allow > >>>>>>>>> certmonger to > >>>>>>>>> talk to the newly defined dogtag ports. Do you have selinux > >>>>>>>>> permissive? > >>>>>>>>> > >>>>>>>>>> There is no man page for pkispawn/pkidestroy :-( According to the > >>>>>>>>>> FHS > >>>>>>>>>> these should not be in /bin but in /usr/sbin (not end-user > >>>>>>>>>> commands). > >>>>>>>>>> > >>>>>>>>> There is a trac ticket open for man pages for pkispawn and > >>>>>>>>> pkidestroy. > >>>>>>>>> We plan to complete this ticket by the time f18 is released. > >>>>>>>>> > >>>>>>>>> We'll look into the location of pkispawn/pkicreate. > >>>>>>>>> > >>>>>>>>>> The output of pkicreate/pkisilent was really terrible and not > >>>>>>>>>> usable at > >>>>>>>>>> all so we didn't display it when failures occurred. It looks like > >>>>>>>>>> that > >>>>>>>>>> has been addressed, at least for the case where a CA is already > >>>>>>>>>> configured and you try to install IPA. Perhaps we should capture > >>>>>>>>>> stderr > >>>>>>>>>> and display that instead of the command-line of pkispawn? > >>>>>>>>>> Again, a > >>>>>>>>>> man > >>>>>>>>>> page would help with the integration. > >>>>>>>>>> > >>>>>>>>>> 2012-09-10T20:51:45Z DEBUG [2/18]: configuring certificate > >>>>>>>>>> server > >>>>>>>>>> instance > >>>>>>>>>> 2012-09-10T20:51:45Z DEBUG args=/bin/pkispawn -s CA -f > >>>>>>>>>> /tmp/tmp_Urraq > >>>>>>>>>> 2012-09-10T20:51:45Z DEBUG stdout= > >>>>>>>>>> 2012-09-10T20:51:45Z DEBUG stderr=pkispawn : ERROR ....... > >>>>>>>>>> PKI > >>>>>>>>>> subsystem 'CA' for instance 'pki-tomcat' already exists! > >>>>>>>>>> > >>>>>>>>>> 2012-09-10T20:51:45Z CRITICAL failed to configure ca instance > >>>>>>>>>> Command > >>>>>>>>>> '/bin/pkispawn -s CA -f /tmp/tmp_Urraq' returned non-zero exit > >>>>>>>>>> status 1 > >>>>>>>>>> > >>>>>>>>> That may be a good idea. Of course. thats an IPA thing, right? > >>>>>>> > >>>>>>> Right, not a show-stopper for this but a new ticket should be > >>>>>>> opened to > >>>>>>> track it. > >>>> > >>>> https://fedorahosted.org/freeipa/ticket/3072 > >>> > >>> Thanks. > >>> > >>> I tested a 2.2 -> 3.0 upgrade with dogtag 9 and it seems to have failed > >>> to restart something: > >>> > >>> [ post rpm -Uvh dist/rpms/freeipa*.rpm ] > >>> > >>> [root@edsel freeipa]# ipa cert-show 1 > >>> ipa: ERROR: Certificate operation cannot be completed: Unable to > >>> communicate with CMS (Service Temporarily Unavailable) > >>> [root@edsel freeipa]# ipactl restart > >>> Restarting Directory Service > >>> Restarting KDC Service > >>> Restarting KPASSWD Service > >>> Restarting MEMCACHE Service > >>> Restarting HTTP Service > >>> Restarting CA Service > >>> [root@edsel freeipa]# ipa cert-show 1 > >>> Certificate: MIIDjTCCAnWgAwIBAgIBATANBgk > >>> ... > >>> > >>> The apache log had this: > >>> > >>> [Tue Sep 11 14:35:42 2012] [error] (111)Connection refused: proxy: AJP: > >>> attempt to connect to 127.0.0.1:9447 (localhost) failed > >>> [Tue Sep 11 14:35:42 2012] [error] ap_proxy_connect_backend disabling > >>> worker for (localhost) > >>> [Tue Sep 11 14:35:42 2012] [error] proxy: AJP: failed to make connection > >>> to backend: localhost > >>> > >>> So I'm gathering that dogtag didn't restart properly, but I'm just > >>> guessing. It could have been the PKI-IPA ds instance too, I'm not sure > >>> where to check. > >>> > >>> I also noticed this in /var/log/ipaupgrade.log: > >>> > >>> 2012-09-11T18:28:22Z DEBUG args=/bin/systemctl start certmonger.service > >>> 2012-09-11T18:28:22Z DEBUG stdout= > >>> 2012-09-11T18:28:22Z DEBUG stderr= > >>> 2012-09-11T18:28:22Z DEBUG args=/bin/systemctl is-active > >>> certmonger.service > >>> 2012-09-11T18:28:22Z DEBUG stdout=active > >>> ... > >>> ... [ snip certutil output ] > >>> ... > >>> 2012-09-11T18:28:52Z DEBUG args=/usr/bin/getcert start-tracking -d > >>> /var/lib/pki-ca/alias -n auditSigningCert cert-pki-ca -c > >>> dogtag-ipa-renew-agent -C /usr/lib64/ipa/certmonger/renew_ca_cert > >>> "auditSigningCert cert-pki-ca" -P XXXXXXXX > >>> 2012-09-11T18:28:52Z DEBUG stdout=Please verify that the certmonger > >>> service is still running. > >>> > >>> 2012-09-11T18:28:52Z DEBUG stderr= > >>> 2012-09-11T18:28:52Z INFO File > >>> "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", > >>> line 614, in run_script > >>> return_value = main_function() > >>> > >>> File "/usr/sbin/ipa-upgradeconfig", line 540, in main > >>> enable_certificate_renewal(api.env.realm) > >>> > >>> File "/usr/sbin/ipa-upgradeconfig", line 455, in > >>> enable_certificate_renewal > >>> ca.configure_renewal() > >>> > >>> File > >>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > >>> 1298, in configure_renewal > >>> self.dogtag_constants.ALIAS_DIR, 'renew_ca_cert "%s"' % nickname) > >>> > >>> File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line > >>> 394, in dogtag_start_tracking > >>> (stdout, stderr, returncode) = ipautil.run(args, nolog=[pin]) > >>> > >>> File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line > >>> 309, in run > >>> raise CalledProcessError(p.returncode, args) > >>> > >>> 2012-09-11T18:28:52Z INFO The ipa-upgradeconfig command failed, > >>> exception: CalledProcessError: Command '/usr/bin/getcert start-tracking > >>> -d /var/lib/pki-ca/alias -n auditSigningCert cert-pki-ca -c > >>> dogtag-ipa-renew-agent -C /usr/lib64/ipa/certmonger/renew_ca_cert > >>> "auditSigningCert cert-pki-ca" -P XXXXXXXX' returned non-zero exit > >>> status 1 > >>> > >>> I'm not sure if this is related to this patch or not. If it isn't can > >>> you file a new ticket on it, or add it the 2.2 -> 3.0 upgrade ticket? > >>> > >>> rob > >> > >> And to reply to myself, how do we imagine that upgrades will work? > >> > >> Is it legal for someone to upgrade to IPA 3.0 and dogtag 10 separately, > >> or do we expect/require it be done at the same time, or one first? > > It's legal to upgrade separately, since we'll support the IPA 3.0 + > Dogtag 9 combination. But to minimize the number of ways things can go > wrong, we could require that IPA is upgraded before Dogtag. > > > Sorry to break this into a ton of e-mails. I updated pki-ca and getting > > a selinux error in permissive: > > > > # /bin/systemctl -a status pki-cad@pki-ca.service > > pki-cad@pki-ca.service - PKI Certificate Authority Server pki-ca > > Loaded: loaded (/usr/lib/systemd/system/pki-cad@.service; > > enabled) > > Active: failed (Result: exit-code) since Tue, 11 Sep 2012 > > 15:36:37 -0400; 3s ago > > Process: 7067 ExecStart=/usr/bin/pkicontrol start ca %i > > (code=exited, status=1/FAILURE) > > Main PID: 5830 (code=exited, status=0/SUCCESS) > > CGroup: name=systemd:/system/pki-cad@.service/pki-ca > > > > Sep 11 15:36:37 edsel.greyoak.com pkicontrol[7067]: /usr/bin/runcon: > > invalid context: system_u:system_r:pki_ca_script_t:s0: Invalid argument > > > > I assume this means I need to re-install pki-selinux? It should be > > possible to tweak the dependencies such that this package is installed > > first. > > I'll look into it, but it would be nice to get some guidance from > someone who knows more about RPMs. > > Its hard to see how this package would not be upgraded first. What seems more likely is that the semodule command loading the pki-selinux module failed to load for some reason during the upgrade.
Is there any indication of what happened from the yum logs? _______________________________________________ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
