Re: [Freeipa-users] CA installation failed on server
Christian Heimes wrote: > On 2015-11-30 16:27, Rob Crittenden wrote: >> Christian Heimes wrote: >>> On 2015-11-30 12:51, Martin Basti wrote: On 28.11.2015 00:14, Rob Crittenden wrote: > Martin Štefany wrote: >> Hello, >> >> I remember experiencing this, but I'm not sure of solution. I think it's >> related to apache (httpd) and his group. >> >> My notes for IPA installation on CentOS 7.x say: >> >> # groupadd -g 48 apache >> # yum -y install ipa-server bind bind-dyndb-ldap >> # usermod -g apache apache >> # ipa-server-install... >> >> CentOS is somehow not creating group apache for apache user and then >> assuming root which is then causing problems with apache later. Pre- >> creating such group before installing httpd and then usermod-ing user >> apache might solve it. >> >> Did you get any warnings while running: >> # yum install -y ipa-server bind bind-dyndb-ldap ? >> >> >> If possible, try installation from scratch with my notes on fresh >> system. If not: >> >> # systemctl stop apache # if it runs >> # groupadd -g 48 apache # I use 48 as apache's UID tends to be also >> 48, or use 'groupadd -r apache' instead >> # usermod -g apache apache >> # ipa-server-install... >> > Sounds unlikely to me. If indeed it did happen you'd need to file a bug > against Apache to create its own uid/gid, which I'm pretty certain it > already does. > > In any case, dogtag doesn't run in Apache so it would be unlikely to > blow up in the CA installer. > > cating the contents of a directory into one log is not at all helpful, > especially given that you missed all the important bits in the > subdirectories beneath it. This is just a mishmash of stuff. We need to > see /var/log/pki/pki-tomcat/ca/debug. > > /var/log/ipaserver-install.log might also be useful to see though it > probably just records in a more verbose way the fact that pkispawn > failed. > > rob > Hello, I see in log this error message: 2015-11-26 08:41:53 pkidestroy : ERROR... subprocess.CalledProcessError: Command '['/usr/bin/sslget', '-n', 'subsystemCert cert-pki-ca', '-p', '272326334956', '-d', '/etc/pki/pki-tomcat/alias', '-e', 'name="/var/lib/pki/pki-tomcat"=CA=caList=ipa.home=443=8443=8443=8443=remove', '-v', '-r', '/ca/agent/ca/updateDomainXML', 'ipa.home:443']' returned non-zero exit status 6! It means that the CA has no been sucessfully uninstalled, and it can cause issues during installation PKI bug: https://fedorahosted.org/pki/ticket/1704 Christian may have workaround (CCed) >>> >>> Hi Martin and Martin, >>> >>> last week I have identified an incompatibility between Dogtag's sslget >>> command and Apache HTTP. sslget sends a server name indication during >>> the TLS/SSL handshake but doesn't send a HTTP Host header. Newer >>> versions of Apache HTTP don't accept requests with SNI and without HTTP >>> Host. You should see a HTTP/400 Bad Request in /var/log/httpd/error_log. >>> >>> The simplest workaround is to bypass Apache and talk to Dogtag directly. >>> In order to do bypass the Apache proxy you have to log onto the server. >>> You also have to become root so you can access the NSS database that >>> contains the private key for authentication. Simply copy and paste the >>> sslget command from the log (everything between "Command '[" and "]' >>> returend non-zero exit status 6!"), remove the comma, replace >>> 'ipa.home:443' with 'ipa.home:8443' and run the command. The command >>> should look like: >>> >>> '/usr/bin/sslget' '-n' 'subsystemCert cert-pki-ca' ... >>> '/ca/agent/ca/updateDomainXML' 'ipa.home:8443' >> >> mod_nss added support for SNI in Fedora 23. It should behave the same >> way as mod_ssl, denying a request that contains an SNI hostname but no >> Host header. >> >> I assume you've filed a bug against dogtag to send a Host header in the >> request? >> >> A better workaround would be to add NSSSNI off to >> /etc/httpd/conf.d/nss.conf within the default VH. Retrying the install >> should work then, or at least get past this sslget error. > > That might explain why the issue hasn't popped up earlier. sslget sets > SNI header without HTTP Host header for a while. Since FreeIPA uses > mod_nss instead of mod_ssl and mod_nss hasn't processed the SNI header > before, sslget didn't fail in the past. > > Yes, I have fixed sslget to send a HTTP Host header with hostname and port. > > No, that won't work. You'd have to change the configuration before you > run uninstall. Install still won't work because uninstall was > incomplete. It's still necessary to remove the Service Domain for the CA > from LDAP. The sslget command call removes it. It's also possible to > remove it with ldapremove over ldapi. The entry is in the >
Re: [Freeipa-users] CA installation failed on server
On 30.11.2015 17:45, Rob Crittenden wrote: Christian Heimes wrote: On 2015-11-30 16:27, Rob Crittenden wrote: Christian Heimes wrote: On 2015-11-30 12:51, Martin Basti wrote: On 28.11.2015 00:14, Rob Crittenden wrote: Martin Štefany wrote: Hello, I remember experiencing this, but I'm not sure of solution. I think it's related to apache (httpd) and his group. My notes for IPA installation on CentOS 7.x say: # groupadd -g 48 apache # yum -y install ipa-server bind bind-dyndb-ldap # usermod -g apache apache # ipa-server-install... CentOS is somehow not creating group apache for apache user and then assuming root which is then causing problems with apache later. Pre- creating such group before installing httpd and then usermod-ing user apache might solve it. Did you get any warnings while running: # yum install -y ipa-server bind bind-dyndb-ldap ? If possible, try installation from scratch with my notes on fresh system. If not: # systemctl stop apache # if it runs # groupadd -g 48 apache # I use 48 as apache's UID tends to be also 48, or use 'groupadd -r apache' instead # usermod -g apache apache # ipa-server-install... Sounds unlikely to me. If indeed it did happen you'd need to file a bug against Apache to create its own uid/gid, which I'm pretty certain it already does. In any case, dogtag doesn't run in Apache so it would be unlikely to blow up in the CA installer. cating the contents of a directory into one log is not at all helpful, especially given that you missed all the important bits in the subdirectories beneath it. This is just a mishmash of stuff. We need to see /var/log/pki/pki-tomcat/ca/debug. /var/log/ipaserver-install.log might also be useful to see though it probably just records in a more verbose way the fact that pkispawn failed. rob Hello, I see in log this error message: 2015-11-26 08:41:53 pkidestroy : ERROR... subprocess.CalledProcessError: Command '['/usr/bin/sslget', '-n', 'subsystemCert cert-pki-ca', '-p', '272326334956', '-d', '/etc/pki/pki-tomcat/alias', '-e', 'name="/var/lib/pki/pki-tomcat"=CA=caList=ipa.home=443=8443=8443=8443=remove', '-v', '-r', '/ca/agent/ca/updateDomainXML', 'ipa.home:443']' returned non-zero exit status 6! It means that the CA has no been sucessfully uninstalled, and it can cause issues during installation PKI bug: https://fedorahosted.org/pki/ticket/1704 Christian may have workaround (CCed) Hi Martin and Martin, last week I have identified an incompatibility between Dogtag's sslget command and Apache HTTP. sslget sends a server name indication during the TLS/SSL handshake but doesn't send a HTTP Host header. Newer versions of Apache HTTP don't accept requests with SNI and without HTTP Host. You should see a HTTP/400 Bad Request in /var/log/httpd/error_log. The simplest workaround is to bypass Apache and talk to Dogtag directly. In order to do bypass the Apache proxy you have to log onto the server. You also have to become root so you can access the NSS database that contains the private key for authentication. Simply copy and paste the sslget command from the log (everything between "Command '[" and "]' returend non-zero exit status 6!"), remove the comma, replace 'ipa.home:443' with 'ipa.home:8443' and run the command. The command should look like: '/usr/bin/sslget' '-n' 'subsystemCert cert-pki-ca' ... '/ca/agent/ca/updateDomainXML' 'ipa.home:8443' mod_nss added support for SNI in Fedora 23. It should behave the same way as mod_ssl, denying a request that contains an SNI hostname but no Host header. I assume you've filed a bug against dogtag to send a Host header in the request? A better workaround would be to add NSSSNI off to /etc/httpd/conf.d/nss.conf within the default VH. Retrying the install should work then, or at least get past this sslget error. That might explain why the issue hasn't popped up earlier. sslget sets SNI header without HTTP Host header for a while. Since FreeIPA uses mod_nss instead of mod_ssl and mod_nss hasn't processed the SNI header before, sslget didn't fail in the past. Yes, I have fixed sslget to send a HTTP Host header with hostname and port. No, that won't work. You'd have to change the configuration before you run uninstall. Install still won't work because uninstall was incomplete. It's still necessary to remove the Service Domain for the CA from LDAP. The sslget command call removes it. It's also possible to remove it with ldapremove over ldapi. The entry is in the cn=CAList,ou=Security Domain,o=ipaca subtree. I'm not sure I follow. You mean my proposed workaround won't work? I think it should because the IPA installer directly tweaks only a few things in nss.conf so if one manually sets NSSSNI then it should be preserved between install/uninstall. I was under the impression this is a fresh install so there is no existing data to deal with. rob If I did read logs right, there was ipa-server-installed, CA uninstallation failed and now IPA server install is
Re: [Freeipa-users] CA installation failed on server
On 2015-11-30 17:48, Martin Basti wrote: > If I did read logs right, there was ipa-server-installed, CA > uninstallation failed and now IPA server install is failing because new > CA cannot be installed due the old instance of CA. Martin, you are right. Daniel didn't mention reinstallation in his initial mail. You and me are aware of the details because we talked to him on IRC. I just asked Daniel on IRC and he confirmed it. Rob couldn't know the fact, hence the misunderstanding. Robert, your workaround fixes uninstallation. But it doesn't fix an already broken system. ipa-server --uninstall leaves the system in an inconsistent state. It removes most of the CA but leaves entries in LDAP ou=Security Domain,o=ipaca. signature.asc Description: OpenPGP digital signature -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] CA installation failed on server
Christian Heimes wrote: > On 2015-11-30 12:51, Martin Basti wrote: >> >> >> On 28.11.2015 00:14, Rob Crittenden wrote: >>> Martin Štefany wrote: Hello, I remember experiencing this, but I'm not sure of solution. I think it's related to apache (httpd) and his group. My notes for IPA installation on CentOS 7.x say: # groupadd -g 48 apache # yum -y install ipa-server bind bind-dyndb-ldap # usermod -g apache apache # ipa-server-install... CentOS is somehow not creating group apache for apache user and then assuming root which is then causing problems with apache later. Pre- creating such group before installing httpd and then usermod-ing user apache might solve it. Did you get any warnings while running: # yum install -y ipa-server bind bind-dyndb-ldap ? If possible, try installation from scratch with my notes on fresh system. If not: # systemctl stop apache # if it runs # groupadd -g 48 apache # I use 48 as apache's UID tends to be also 48, or use 'groupadd -r apache' instead # usermod -g apache apache # ipa-server-install... >>> Sounds unlikely to me. If indeed it did happen you'd need to file a bug >>> against Apache to create its own uid/gid, which I'm pretty certain it >>> already does. >>> >>> In any case, dogtag doesn't run in Apache so it would be unlikely to >>> blow up in the CA installer. >>> >>> cating the contents of a directory into one log is not at all helpful, >>> especially given that you missed all the important bits in the >>> subdirectories beneath it. This is just a mishmash of stuff. We need to >>> see /var/log/pki/pki-tomcat/ca/debug. >>> >>> /var/log/ipaserver-install.log might also be useful to see though it >>> probably just records in a more verbose way the fact that pkispawn >>> failed. >>> >>> rob >>> >> Hello, >> >> I see in log this error message: >> >> 2015-11-26 08:41:53 pkidestroy : ERROR... >> subprocess.CalledProcessError: Command '['/usr/bin/sslget', '-n', >> 'subsystemCert cert-pki-ca', '-p', '272326334956', '-d', >> '/etc/pki/pki-tomcat/alias', '-e', >> 'name="/var/lib/pki/pki-tomcat"=CA=caList=ipa.home=443=8443=8443=8443=remove', >> '-v', '-r', '/ca/agent/ca/updateDomainXML', 'ipa.home:443']' returned >> non-zero exit status 6! >> >> It means that the CA has no been sucessfully uninstalled, and it can >> cause issues during installation >> PKI bug: >> https://fedorahosted.org/pki/ticket/1704 >> >> Christian may have workaround (CCed) > > Hi Martin and Martin, > > last week I have identified an incompatibility between Dogtag's sslget > command and Apache HTTP. sslget sends a server name indication during > the TLS/SSL handshake but doesn't send a HTTP Host header. Newer > versions of Apache HTTP don't accept requests with SNI and without HTTP > Host. You should see a HTTP/400 Bad Request in /var/log/httpd/error_log. > > The simplest workaround is to bypass Apache and talk to Dogtag directly. > In order to do bypass the Apache proxy you have to log onto the server. > You also have to become root so you can access the NSS database that > contains the private key for authentication. Simply copy and paste the > sslget command from the log (everything between "Command '[" and "]' > returend non-zero exit status 6!"), remove the comma, replace > 'ipa.home:443' with 'ipa.home:8443' and run the command. The command > should look like: > > '/usr/bin/sslget' '-n' 'subsystemCert cert-pki-ca' ... > '/ca/agent/ca/updateDomainXML' 'ipa.home:8443' mod_nss added support for SNI in Fedora 23. It should behave the same way as mod_ssl, denying a request that contains an SNI hostname but no Host header. I assume you've filed a bug against dogtag to send a Host header in the request? A better workaround would be to add NSSSNI off to /etc/httpd/conf.d/nss.conf within the default VH. Retrying the install should work then, or at least get past this sslget error. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] CA installation failed on server
On 2015-11-30 16:27, Rob Crittenden wrote: > Christian Heimes wrote: >> On 2015-11-30 12:51, Martin Basti wrote: >>> >>> >>> On 28.11.2015 00:14, Rob Crittenden wrote: Martin Štefany wrote: > Hello, > > I remember experiencing this, but I'm not sure of solution. I think it's > related to apache (httpd) and his group. > > My notes for IPA installation on CentOS 7.x say: > > # groupadd -g 48 apache > # yum -y install ipa-server bind bind-dyndb-ldap > # usermod -g apache apache > # ipa-server-install... > > CentOS is somehow not creating group apache for apache user and then > assuming root which is then causing problems with apache later. Pre- > creating such group before installing httpd and then usermod-ing user > apache might solve it. > > Did you get any warnings while running: > # yum install -y ipa-server bind bind-dyndb-ldap ? > > > If possible, try installation from scratch with my notes on fresh > system. If not: > > # systemctl stop apache # if it runs > # groupadd -g 48 apache # I use 48 as apache's UID tends to be also > 48, or use 'groupadd -r apache' instead > # usermod -g apache apache > # ipa-server-install... > Sounds unlikely to me. If indeed it did happen you'd need to file a bug against Apache to create its own uid/gid, which I'm pretty certain it already does. In any case, dogtag doesn't run in Apache so it would be unlikely to blow up in the CA installer. cating the contents of a directory into one log is not at all helpful, especially given that you missed all the important bits in the subdirectories beneath it. This is just a mishmash of stuff. We need to see /var/log/pki/pki-tomcat/ca/debug. /var/log/ipaserver-install.log might also be useful to see though it probably just records in a more verbose way the fact that pkispawn failed. rob >>> Hello, >>> >>> I see in log this error message: >>> >>> 2015-11-26 08:41:53 pkidestroy : ERROR... >>> subprocess.CalledProcessError: Command '['/usr/bin/sslget', '-n', >>> 'subsystemCert cert-pki-ca', '-p', '272326334956', '-d', >>> '/etc/pki/pki-tomcat/alias', '-e', >>> 'name="/var/lib/pki/pki-tomcat"=CA=caList=ipa.home=443=8443=8443=8443=remove', >>> '-v', '-r', '/ca/agent/ca/updateDomainXML', 'ipa.home:443']' returned >>> non-zero exit status 6! >>> >>> It means that the CA has no been sucessfully uninstalled, and it can >>> cause issues during installation >>> PKI bug: >>> https://fedorahosted.org/pki/ticket/1704 >>> >>> Christian may have workaround (CCed) >> >> Hi Martin and Martin, >> >> last week I have identified an incompatibility between Dogtag's sslget >> command and Apache HTTP. sslget sends a server name indication during >> the TLS/SSL handshake but doesn't send a HTTP Host header. Newer >> versions of Apache HTTP don't accept requests with SNI and without HTTP >> Host. You should see a HTTP/400 Bad Request in /var/log/httpd/error_log. >> >> The simplest workaround is to bypass Apache and talk to Dogtag directly. >> In order to do bypass the Apache proxy you have to log onto the server. >> You also have to become root so you can access the NSS database that >> contains the private key for authentication. Simply copy and paste the >> sslget command from the log (everything between "Command '[" and "]' >> returend non-zero exit status 6!"), remove the comma, replace >> 'ipa.home:443' with 'ipa.home:8443' and run the command. The command >> should look like: >> >> '/usr/bin/sslget' '-n' 'subsystemCert cert-pki-ca' ... >> '/ca/agent/ca/updateDomainXML' 'ipa.home:8443' > > mod_nss added support for SNI in Fedora 23. It should behave the same > way as mod_ssl, denying a request that contains an SNI hostname but no > Host header. > > I assume you've filed a bug against dogtag to send a Host header in the > request? > > A better workaround would be to add NSSSNI off to > /etc/httpd/conf.d/nss.conf within the default VH. Retrying the install > should work then, or at least get past this sslget error. That might explain why the issue hasn't popped up earlier. sslget sets SNI header without HTTP Host header for a while. Since FreeIPA uses mod_nss instead of mod_ssl and mod_nss hasn't processed the SNI header before, sslget didn't fail in the past. Yes, I have fixed sslget to send a HTTP Host header with hostname and port. No, that won't work. You'd have to change the configuration before you run uninstall. Install still won't work because uninstall was incomplete. It's still necessary to remove the Service Domain for the CA from LDAP. The sslget command call removes it. It's also possible to remove it with ldapremove over ldapi. The entry is in the cn=CAList,ou=Security Domain,o=ipaca subtree. Christian signature.asc Description: OpenPGP digital signature -- Manage your subscription for the Freeipa-users
Re: [Freeipa-users] CA installation failed on server
Christian Heimes wrote: > On 2015-11-30 17:48, Martin Basti wrote: >> If I did read logs right, there was ipa-server-installed, CA >> uninstallation failed and now IPA server install is failing because new >> CA cannot be installed due the old instance of CA. > > Martin, you are right. Daniel didn't mention reinstallation in his > initial mail. You and me are aware of the details because we talked to > him on IRC. I just asked Daniel on IRC and he confirmed it. Rob couldn't > know the fact, hence the misunderstanding. > > Robert, your workaround fixes uninstallation. But it doesn't fix an > already broken system. ipa-server --uninstall leaves the system in an > inconsistent state. It removes most of the CA but leaves entries in LDAP > ou=Security Domain,o=ipaca. I don't know quite what to say. Any already broken system? The IPA installer isn't (yet) idempotent so any failure installing needs to be uninstalled, corrected, and tried again. There is no powering onwards. Left over data? In a standalone system this is moot because IPA needs to start as a clean slate for a new installation attempt. I really don't see how my workaround made any difference at all in the uninstall since it should just be removing bits not doing anything over LDAP. And in any case, uninstalling the server also wipes out the LDAP server so it's a moot point. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] CA installation failed on server
On 28.11.2015 00:14, Rob Crittenden wrote: Martin Štefany wrote: Hello, I remember experiencing this, but I'm not sure of solution. I think it's related to apache (httpd) and his group. My notes for IPA installation on CentOS 7.x say: # groupadd -g 48 apache # yum -y install ipa-server bind bind-dyndb-ldap # usermod -g apache apache # ipa-server-install... CentOS is somehow not creating group apache for apache user and then assuming root which is then causing problems with apache later. Pre- creating such group before installing httpd and then usermod-ing user apache might solve it. Did you get any warnings while running: # yum install -y ipa-server bind bind-dyndb-ldap ? If possible, try installation from scratch with my notes on fresh system. If not: # systemctl stop apache # if it runs # groupadd -g 48 apache # I use 48 as apache's UID tends to be also 48, or use 'groupadd -r apache' instead # usermod -g apache apache # ipa-server-install... Sounds unlikely to me. If indeed it did happen you'd need to file a bug against Apache to create its own uid/gid, which I'm pretty certain it already does. In any case, dogtag doesn't run in Apache so it would be unlikely to blow up in the CA installer. cating the contents of a directory into one log is not at all helpful, especially given that you missed all the important bits in the subdirectories beneath it. This is just a mishmash of stuff. We need to see /var/log/pki/pki-tomcat/ca/debug. /var/log/ipaserver-install.log might also be useful to see though it probably just records in a more verbose way the fact that pkispawn failed. rob Hello, I see in log this error message: 2015-11-26 08:41:53 pkidestroy : ERROR... subprocess.CalledProcessError: Command '['/usr/bin/sslget', '-n', 'subsystemCert cert-pki-ca', '-p', '272326334956', '-d', '/etc/pki/pki-tomcat/alias', '-e', 'name="/var/lib/pki/pki-tomcat"=CA=caList=ipa.home=443=8443=8443=8443=remove', '-v', '-r', '/ca/agent/ca/updateDomainXML', 'ipa.home:443']' returned non-zero exit status 6! It means that the CA has no been sucessfully uninstalled, and it can cause issues during installation PKI bug: https://fedorahosted.org/pki/ticket/1704 Christian may have workaround (CCed) Martin -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
