On 06/21/2018 08:57 AM, Jokinen Eemeli via FreeIPA-users wrote:
Hi!

Forgot kinit:

--
kinit admin
Password for admin@<<REALM>>:
klist
Ticket cache: KEYRING:persistent:0:0
Default principal: admin@<<REALM>>

Valid starting       Expires              Service principal
06/21/2018 09:55:07  06/22/2018 09:54:54  HTTP/<<ipa1.fqdn>>@<<REALM>>
06/21/2018 09:55:02  06/22/2018 09:54:54  krbtgt/<<REALM>>@<<REALM>>
ipa config-show
ipa: ERROR: No valid Negotiate header in server response
--

Hi,
can you check if the service gssproxy is running:
# systemctl status gssproxy
and if not, restart it.

Flo

Still no luck getting ipa config


Eemeli

-----Original Message-----
From: Florence Blanc-Renaud [mailto:[email protected]]
Sent: torstai 21. kesäkuuta 2018 9.43
To: FreeIPA users list <[email protected]>
Cc: Jokinen Eemeli <[email protected]>
Subject: Re: [Freeipa-users] Re: Problems after IPA upgrade: ipa-server-upgrade 
doesn't complete, pki-tomcatd won't start

On 06/21/2018 07:34 AM, Jokinen Eemeli via FreeIPA-users wrote:
Hi!

We do have 2 IPA nodes configured but the second node has been down for some 
time. Tried to update it to same version as node1:
- Won't start tells me to use ipa-server-upgrade
- Ipa-server-upgrade fails at start, doesn't start directory server
-  /var/log/dirsrv/slapd-<<REALM>>/errors log has some acl_parse
errors but last row tells me
--
EMERG - main - Fatal Error---No ports specified. Exiting now.
--

Hi,

in this case let's concentrate on node1.

On node1 ipa config-show results only
--
ipa: ERROR: did not receive Kerberos credentials
--

You need to run kinit to get kerberos credentials before any ipa * command. If 
the ipa stack is stopped because pki-tomcat refused to start, you can force the 
start up, ignoring failed services:
$ sudo ipactl start --ignore-service-failures

Then to check who is renewal master:
$ sudo kinit admin
$ sudo ipa config-show

If node1 is not the renewal master, make it the renewal master from now on 
(follow instructions from 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/linux_domain_identity_authentication_and_policy_guide/server-roles#server-roles-promote-to-ca).
After this step, check the certificates in the NSS databases vs the 
certificates stored in LDAP.

HTH,
Flo



Eemeli

-----Original Message-----
From: Florence Blanc-Renaud [mailto:[email protected]]
Sent: keskiviikko 20. kesäkuuta 2018 19.49
To: FreeIPA users list <[email protected]>
Cc: Jokinen Eemeli <[email protected]>
Subject: Re: [Freeipa-users] Problems after IPA upgrade:
ipa-server-upgrade doesn't complete, pki-tomcatd won't start

On 06/20/2018 01:53 PM, Jokinen Eemeli via FreeIPA-users wrote:
Hello all!

I have very similiar problem as this one:

https://lists.fedorahosted.org/archives/list/[email protected]
r ahosted.org/thread/YU6TZHOJAV5QHHHPQWJHYX3FP4OHA37X/

ipa-server-upgrade fails as below

--

Update complete

Upgrading IPA services

Upgrading the configuration of the IPA services

[Verifying that root certificate is published]

[Migrate CRL publish directory]

CRL tree already moved

[Verifying that CA proxy configuration is correct]

IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run
command ipa-server-upgrade manually.

CA did not start in 300.0s

The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log
for more information

--

And the log tells me that CA returns status 500

--

DEBUG Waiting for CA to start...

DEBUG request POST http://<<ipa1.fqdn>>:8080/ca/admin/ca/getStatus
<http://%3c%3cipa1.fqdn%3e%3e:8080/ca/admin/ca/getStatus>

DEBUG request body ''

DEBUG response status 500

DEBUG response headers Server: Apache-Coyote/1.1

Content-Type: text/html;charset=utf-8

Content-Language: en

Content-Length: 2208

Date: Fri, 15 Jun 2018 10:05:29 GMT

Connection: close

DEBUG response body '<html><head><title>Apache Tomcat/7.0.76 - Error
report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#52
5
D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#52
5
D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#52
5
D76;font-size:14px;}
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:whi
t
e;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#52
5
D76;}
P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;fon
t -size:12px;}A {color : black;}A.name {color : black;}HR {color :
#525D76;}--></style> </head><body><h1>HTTP Status 500 - Subsystem
unavailable</h1><HR size="1" noshade="noshade"><p><b>type</b>
Exception report</p><p><b>message</b> <u>Subsystem
unavailable</u></p><p><b>description</b> <u>The server encountered an
internal error that prevented it from fulfilling this
request.</u></p><p><b>exception</b>
<pre>javax.ws.rs.ServiceUnavailableException: Subsystem
unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstra
i
nts(ProxyRealm.java:145)\n\torg.apache.catalina.authenticator.Authent
i
catorBase.invoke(AuthenticatorBase.java:500)\n\torg.apache.catalina.v
a
lves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\torg.apache.
catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)\n\torg.
apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:44
5
)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(Abstrac
t
Http11Processor.java:1087)\n\torg.apache.coyote.AbstractProtocol$Abst
r
actConnectionHandler.process(AbstractProtocol.java:637)\n\torg.apache.
tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
\
n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
o
r.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(Th
r
eadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThrea
d
$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thr
e ad.java:748)\n</pre></p><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache
Tomcat/7.0.76 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache
Tomcat/7.0.76</h3></body></html>'

DEBUG The CA status is: check interrupted due to error: Retrieving CA
status failed with status 500

DEBUG Waiting for CA to start...

ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and
run command ipa-server-upgrade manually.

File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line
172, in execute

       return_value = self.run()

     File
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrad
e
.py",
line 48, in run

       raise admintool.ScriptError(str(e))

The ipa-server-upgrade command failed, exception: ScriptError: CA did
not start in 300.0s

ERROR CA did not start in 300.0s

--

With command "ipactl start --ignore-service-failures" I can start all
the services but pki-tomcatd.

--

Directory Service: RUNNING

krb5kdc Service: RUNNING

kadmin Service: RUNNING

named Service: RUNNING

httpd Service: RUNNING

pki-tomcatd Service: STOPPED

smb Service: RUNNING

winbind Service: RUNNING

ipa-otpd Service: RUNNING

ipa-dnskeysyncd Service: RUNNING

ipa: INFO: The ipactl command was successful

--

Suggested resolution to above problem doesn't help me since the LDAP
and NSS DB seem to have same certificates (some difference in
wrapping but the string is same if I take out the line breaks) and
even the serial number matches.

--

certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca' -a

-----BEGIN CERTIFICATE-----

MIIDjD.

.Prh2G

-----END CERTIFICATE-----

certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca'
|grep Serial

           Serial Number: 4 (0x4)

ldapsearch -LLL -D 'cn=directory manager' -W -b
uid=pkidbuser,ou=people,o=ipaca userCertificate description seeAlso

Enter LDAP Password:

dn: uid=pkidbuser,ou=people,o=ipaca

userCertificate:: MIIDjD.

.Prh2

G

description: 2;4;CN=Certificate Authority,O=<<REALM>>;CN=CA
Subsystem,

O=<<REALM>>

seeAlso: CN=CA Subsystem,O=<<REALM>>

--

And here's where my actual knowledge of things end. I've been trying
to figure out all kind of logs (tomcat, Kerberos, directory server,
.) but haven't found a solid reason for it. I'm starting to believe
this is a certificate issue, because although "getcert list" tells me
that the certificate status is "Monitoring" on all certificates the
expiry date is already in the past (current date 20.6.2018,
certificate expiry
21.03.2018) on 4 certificates and it won't update even if I resubmit
it or delete certificate and manually redo it (it got the same date
as the "old ones"). The "main certs" ("caSigningCert cert-pki-ca",
"Server-Cert cert-pki-ca" and two directory server certs) are valid
for years (until
2020+).

--

Request ID '20160331084234':

           status: MONITORING

           stuck: no

           key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigning
C ert cert-pki-ca',token='NSS Certificate DB',pin set

           certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigning
C ert cert-pki-ca',token='NSS Certificate DB'

           CA: dogtag-ipa-ca-renew-agent

           issuer: CN=Certificate Authority,O=<<REALM>>

           subject: CN=OCSP Subsystem,O=<<REALM>>

           expires: 2018-03-21 09:42:04 UTC

           key usage:
digitalSignature,nonRepudiation,keyCertSign,cRLSign

           eku: id-kp-OCSPSigning

           pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

           post-save command:
/usr/libexec/ipa/certmonger/renew_ca_cert
"ocspSigningCert cert-pki-ca"

           track: yes

           auto-renew: yes

Request ID '20160331085008':

           status: MONITORING

           stuck: no

           key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='
N SS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

           certificate:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='
N
SS
Certificate DB'

           CA: IPA

           issuer: CN=Certificate Authority,O=<<REALM>>

           subject: CN=<<ipasrv1.fqdn>>,O=<<REALM>>

           expires: 2020-03-04 09:58:23 UTC

           principal name: HTTP/<<ipasrv1.fqdn>>@<<REALM>>

           key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

           eku: id-kp-serverAuth,id-kp-clientAuth

           pre-save command:

           post-save command: /usr/lib64/ipa/certmonger/restart_httpd

           track: yes

           auto-renew: yes

--

Has anyone else bumped into same kind of issues? Any ideas where I
should continue looking? I'm starting to run out of ideas.

Eemeli Jokinen



_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to
[email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
o rahosted.org/message/5XER2RAII4UH5URIMPL3GFHVBD7B6YSM/


Hi,

does your topology include multiple CA instances? You need first to find which 
master is the CA renewal master:
ipa config-show | grep "renewal master"

On this host, check that the certificates are still valid and consistent with 
the content of the LDAP entries. If it is not the case, you need to repair the 
CA renewal master first.

When the CA renewal master is OK, check if the replication is working with the 
other CA instances, and repair the other masters.
HTH,
Flo
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to
[email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
rahosted.org/message/PZTT22YGPX2567V7EL7LUFLEO4ZKOH6U/


_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/[email protected]/message/7COMUEBZFYX6KV55HWWNFZ2H6JKDTNVB/

_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/[email protected]/message/42LF6KNFDJ5B47FF75MJAUEFJGWRITGI/

Reply via email to