Boris wrote: > woa. It worked. Thanks a lot! > Now I can step by step upgrade to the latest fedora. > > Do I need to stop all IPA instances to reset the directory manager password?
Yes. And you need to reset it on all servers. The value is not replicated. Follow the KCS carefully. rob > > Am Do., 20. Feb. 2025 um 18:56 Uhr schrieb Florence Blanc-Renaud > <[email protected] <mailto:[email protected]>>: > > Hi, > > I think I spotted the issue: https://pagure.io/freeipa/issue/9381 > > It was fixed on the ipa-4-10 branch but never released in ipa 4.10. > Since your ipa1 host has freeipa-server-4.10.1-1.fc37 it doesn't > have the patch. > > Check if you have a drop-in file > > /etc/systemd/system/[email protected]/ipa.conf > > If not, create one with the following content: > > # cat /etc/systemd/system/[email protected]/ipa.conf > [Service] > Environment=LC_ALL=C.UTF-8 > ExecStartPost=/usr/libexec/ipa/ipa-pki-wait-running > > then launch systemctl --system daemon-reload and ipa-server-upgrade. > > HTH, > flo > > On Thu, Feb 20, 2025 at 5:29 PM Boris <[email protected] > <mailto:[email protected]>> wrote: > > Hey, > > *pki-server subsystem-show ca *and *curl --cert > /var/lib/ipa/ra-agent.pem --key /var/lib/ipa/ra-agent.key > https://`hostname`:8443/ca/rest/account/login* > gave the expected results. > > My thought was that there is no ca available during the update, > and thats why I wanted to add the 2nd host as CA. > > I feel a bit nervous about posting both logs, because it feels > hard to clean them up from some information. > I gave my best. You can find both logs here: > https://blktrace.kervyn.de/debug.2025-02-20.log.gz > https://blktrace.kervyn.de/ipaupgrade.log.gz > > > > Am Do., 20. Feb. 2025 um 16:53 Uhr schrieb Florence Blanc-Renaud > <[email protected] <mailto:[email protected]>>: > > Hi, > > On Wed, Feb 19, 2025 at 11:57 AM Boris <[email protected] > <mailto:[email protected]>> wrote: > > Hi flo, > > `ipa cert-show 1` works on both IPA hosts and returns > correct data, from what I can tell. > > That's strange because cert-show is also authenticating to > the CA REST API. > > > `ipa config-show` gies the following: > IPA masters: ipa1.redacted, ipa2.redacted > IPA master capable of PKINIT: ipa2.redacted > IPA CA servers: ipa1.redacted > IPA CA renewal master: ipa1.redacted > IPA DNS servers: ipa1.redacted, ipa2.redacted > > regarding the named crashes: I think the problem might > be related to ldap. The last time the named daemons were > in a restart/crash loop I restarted the ipa2 host which > immediately resolved the problem. > ipa1: > bind-9.18.19-1.fc37.x86_64 > bind-dyndb-ldap-11.10-17.fc37.x86_64 > > ipa2: > bind-9.16.28-1.fc35.x86_64 > bind-dyndb-ldap-11.9-12.fc35.x86_64 > > for the coredump I would need your guidance what to do, > because I am not that firm with named debugging. > > Here are the last couple of line from the > /var/log/ipaupgrade.log file. The update seems to go > through, but it fails when it needs to authenticate with > the CA REST API > > 2025-01-21T20:24:14Z DEBUG stderr= > 2025-01-21T20:24:14Z DEBUG Starting external process > 2025-01-21T20:24:14Z DEBUG args=['/bin/systemctl', > 'start', 'certmonger.service'] > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0 > 2025-01-21T20:24:15Z DEBUG stdout= > 2025-01-21T20:24:15Z DEBUG stderr= > 2025-01-21T20:24:15Z DEBUG Starting external process > 2025-01-21T20:24:15Z DEBUG args=['/bin/systemctl', > 'is-active', 'certmonger.service'] > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0 > 2025-01-21T20:24:15Z DEBUG stdout=active > > 2025-01-21T20:24:15Z DEBUG stderr= > 2025-01-21T20:24:15Z DEBUG Start of certmonger.service > complete > 2025-01-21T20:24:15Z DEBUG Starting external process > 2025-01-21T20:24:15Z DEBUG args=['pki-server', > 'subsystem-show', 'kra'] > 2025-01-21T20:24:15Z DEBUG Process finished, return code=1 > 2025-01-21T20:24:15Z DEBUG stdout= > 2025-01-21T20:24:15Z DEBUG stderr=ERROR: ERROR: No kra > subsystem in instance pki-tomcat. > > 2025-01-21T20:24:15Z INFO [Update certmonger certificate > renewal configuration] > 2025-01-21T20:24:15Z DEBUG Loading Index file from > '/var/lib/ipa/sysrestore/sysrestore.index' > 2025-01-21T20:24:15Z DEBUG Starting external process > 2025-01-21T20:24:15Z DEBUG args=['/usr/bin/certutil', > '-d', 'sql:/etc/dirsrv/slapd-redacted/', '-L', '-n', > 'Server-Cert', '-a', '-f', > '/etc/dirsrv/slapd-redacted/pwdfile.txt'] > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0 > 2025-01-21T20:24:15Z DEBUG stdout=-----BEGIN > CERTIFICATE----- > redacted > -----END CERTIFICATE----- > > 2025-01-21T20:24:15Z DEBUG stderr= > 2025-01-21T20:24:15Z DEBUG Loading Index file from > '/var/lib/ipa/sysrestore/sysrestore.index' > 2025-01-21T20:24:15Z DEBUG Starting external process > 2025-01-21T20:24:15Z DEBUG args=['/usr/bin/certutil', > '-d', 'sql:/etc/pki/pki-tomcat/alias', '-L', '-f', > '/etc/pki/pki-tomcat/alias/pwdfile.txt'] > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0 > 2025-01-21T20:24:15Z DEBUG stdout= > Certificate Nickname > Trust Attributes > > SSL,S/MIME,JAR/XPI > > caSigningCert cert-pki-ca > CTu,Cu,Cu > caSigningCert cert-pki-ca > 6148bb27-6bd6-4a0a-b607-6ba538a6c401 u,u,u > ocspSigningCert cert-pki-ca > u,u,u > subsystemCert cert-pki-ca > u,u,u > auditSigningCert cert-pki-ca > u,u,Pu > Server-Cert cert-pki-ca > u,u,u > > 2025-01-21T20:24:15Z DEBUG stderr= > 2025-01-21T20:24:15Z INFO Certmonger certificate renewal > configuration already up-to-date > 2025-01-21T20:24:15Z INFO [Enable PKIX certificate path > discovery and validation] > 2025-01-21T20:24:15Z DEBUG Loading StateFile from > '/var/lib/ipa/sysupgrade/sysupgrade.state' > 2025-01-21T20:24:15Z INFO PKIX already enabled > 2025-01-21T20:24:15Z INFO [Authorizing RA Agent to > modify profiles] > 2025-01-21T20:24:15Z INFO [Authorizing RA Agent to > manage lightweight CAs] > 2025-01-21T20:24:15Z INFO [Ensuring Lightweight CAs > container exists in Dogtag database] > 2025-01-21T20:24:15Z INFO [Adding default OCSP URI > configuration] > 2025-01-21T20:24:15Z INFO [Disabling cert publishing] > 2025-01-21T20:24:15Z INFO [Ensuring CA is using > LDAPProfileSubsystem] > 2025-01-21T20:24:15Z INFO [Migrating certificate > profiles to LDAP] > 2025-01-21T20:24:15Z DEBUG Profile 'AdminCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'DomainController' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'ECAdminCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'acmeServerCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caAdminCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caAgentFileSigning' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caAgentServerCert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caAuditSigningCert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caCACert' is already > in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCECUserCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCECserverCert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile > 'caCMCECsubsystemCert' is already in LDAP and enabled; > skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCUserCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caCrossSignedCACert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caDirBasedDualCert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caDirPinUserCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caDirUserCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caDirUserRenewal' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caDualCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caDualRAuserCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caECAdminCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caECAgentServerCert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caECDirPinUserCert' > is already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caECDirUserCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z DEBUG Profile 'caECDualCert' is > already in LDAP and enabled; skipping > 2025-01-21T20:24:15Z INFO Migrating profile > 'caECFullCMCSharedTokenCert' > 2025-01-21T20:24:15Z DEBUG request GET > https://ipa1.redacted:8443/ca/rest/account/login > 2025-01-21T20:24:15Z DEBUG request body '' > 2025-01-21T20:24:16Z DEBUG response status 404 > 2025-01-21T20:24:16Z DEBUG response headers > Content-Type: text/html;charset=utf-8 > Content-Language: en > Content-Length: 784 > Date: Tue, 21 Jan 2025 20:24:16 GMT > > > 2025-01-21T20:24:16Z DEBUG response body (decoded): > b'<!doctype html><html lang="en"><head><title>HTTP > Status 404 \xe2\x80\x93 Not Found</title><style > type="text/css">body > {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b > {color:white;background-color:#525D76;} h1 > {font-size:22px;} h2 {font-size:16px;} h3 > {font-size:14px;} p {font-size:12px;} a {color:black;} > .line > > {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP > Status 404 \xe2\x80\x93 Not Found</h1><hr class="line" > /><p><b>Type</b> Status Report</p><p><b>Message</b> The > requested resource > [/ca/rest/account/login] is not > available</p><p><b>Description</b> The origin server did > not find a current representation for the target > resource or is not willing to disclose that one > exists.</p><hr class="line" /><h3>Apache > Tomcat/9.0.82</h3></body></html>' > 2025-01-21T20:24:16Z ERROR IPA server upgrade failed: > Inspect /var/log/ipaupgrade.log and run command > ipa-server-upgrade manually. > 2025-01-21T20:24:16Z DEBUG File > "/usr/lib/python3.11/site-packages/ipapython/admintool.py", > line 180, in execute > return_value = self.run() > ^^^^^^^^^^ > File > > "/usr/lib/python3.11/site-packages/ipaserver/install/ipa_server_upgrade.py", > line 54, in run > server.upgrade() > File > > "/usr/lib/python3.11/site-packages/ipaserver/install/server/upgrade.py", > line 2061, in upgrade > upgrade_configuration() > File > > "/usr/lib/python3.11/site-packages/ipaserver/install/server/upgrade.py", > line 1914, in upgrade_configuration > ca_enable_ldap_profile_subsystem(ca) > File > > "/usr/lib/python3.11/site-packages/ipaserver/install/server/upgrade.py", > line 458, in ca_enable_ldap_profile_subsystem > cainstance.migrate_profiles_to_ldap() > File > > "/usr/lib/python3.11/site-packages/ipaserver/install/cainstance.py", > line 2155, in migrate_profiles_to_ldap > _create_dogtag_profile(profile_id, profile_data, > overwrite=False) > File > > "/usr/lib/python3.11/site-packages/ipaserver/install/cainstance.py", > line 2209, in _create_dogtag_profile > with api.Backend.ra_certprofile as profile_api: > File > > "/usr/lib/python3.11/site-packages/ipaserver/plugins/dogtag.py", > line 1211, in __enter__ > raise errors.RemoteRetrieveError(reason=_('Failed to > authenticate to CA REST API')) > > Can you check if the CA subsystem is enabled? > # *pki-server subsystem-show ca* > Subsystem ID: ca > Instance ID: pki-tomcat > Enabled: True > > If yes, try to authenticate to the rest API with curl: > # *curl --cert /var/lib/ipa/ra-agent.pem --key > /var/lib/ipa/ra-agent.key > https://`hostname`:8443/ca/rest/account/login* > {"id":"ipara","FullName":"ipara","Roles":["Certificate > Manager Agents","Enterprise ACME > Administrators","Registration Manager Agents","Security > Domain Administrators"],"Attributes":{"Attribute":[]}} > > If the above commands are working, retry the upgrade with > # *ipa-server-upgrade* > and send us the full /var/log/ipaupgrade.log and > /var/log/pki/pki-tomcat/ca/debug.$DATE.log. > > flo > > 2025-01-21T20:24:16Z DEBUG The ipa-server-upgrade > command failed, exception: RemoteRetrieveError: Failed > to authenticate to CA REST API > 2025-01-21T20:24:16Z ERROR Unexpected error - see > /var/log/ipaupgrade.log for details: > RemoteRetrieveError: Failed to authenticate to CA REST API > 2025-01-21T20:24:16Z ERROR The ipa-server-upgrade > command failed. See /var/log/ipaupgrade.log for more > information > > Am Mi., 19. Feb. 2025 um 10:56 Uhr schrieb Florence > Blanc-Renaud <[email protected] <mailto:[email protected]>>: > > Hi, > > in a previous message you mentioned that the > directory manager password is lost. You can follow > this article to reset the DM password: > https://access.redhat.com/solutions/203473 > > Named crashes could be related to multiple issues: > - inconsistent versions between bind and > bind-dyndb-ldap. Which versions do you have? > - an insufficient number of threads > - an issue when reloading the zones > If you can gather a coredump and install the debug > packages it could help identify if you're hitting a > known issue. > > You mentioned that ipa1 needs to be started with > --force, can you tell which service is failing and > provide the logs? There should be also more > information in /var/log/ipaupgrade.log. > > In order to check the CA state, a useful command is > 'ipa cert-show 1' as it communicates with the CA to > gather the certificate details. If this command is > failing (likely with "Failed to Authenticate to CA > rest API") you need to understand where the config > is broken. > Start by checking which system is the CA renewal master: > ipa config-show > > The CA renewal master will be your priority. > > flo > > On Wed, Feb 19, 2025 at 10:25 AM Boris via > FreeIPA-users <[email protected] > <mailto:[email protected]>> wrote: > > I think the CA is working, but I don't know for > sure and how to verify it. At least there are no > expired certs on both ipa hosts > > [root@ipa1 ~]# getcert list | grep expires > expires: 2025-11-29 13:19:40 CET > expires: 2025-04-15 16:27:34 CEST > expires: 2025-04-15 16:26:44 CEST > expires: 2025-04-15 16:27:14 CEST > expires: 2037-08-19 16:11:12 CEST > expires: 2025-04-15 16:27:54 CEST > expires: 2025-04-15 16:27:04 CEST > expires: 2040-02-12 12:46:50 CET > expires: 2025-05-29 16:12:51 CEST > expires: 2026-01-26 13:48:23 CET > > [root@ipa2 ~]# getcert list | grep expires > expires: 2027-02-16 10:42:29 CET > expires: 2027-02-16 10:42:51 CET > expires: 2025-04-15 16:27:04 CEST > expires: 2027-02-16 10:43:26 CET > > The healthcheck showed some "group is not > correct" and "files are to permissive" which I > resolved. > Now I have these to checks which do not tell me > anything > "msg": "certmonger tracking request {key} > found and is not expected on an IPA master." > "msg": "No KDC workers defined in {sysconfig}" > > Am Di., 18. Feb. 2025 um 15:22 Uhr schrieb Rob > Crittenden via FreeIPA-users > <[email protected] > <mailto:[email protected]>>: > > Boris wrote: > > Hi Rob, > > > > I have two hosts: ipa1 and ipa2 > > > > ipa1: > > Fedora 37 > > freeipa-server-4.10.1-1.fc37.x86_64 > > Managed suffixes: domain, ca > > running with ipactl start --force because > the update is not working (The > > ipa-server-upgrade command failed, > exception: RemoteRetrieveError: > > Failed to authenticate to CA REST API). > > I tried to upgrade, but the upgrade did > not go through. > > Your existing CA is having issues. I'd start > by checking that your CA > certificates are still valid: getcert list | > grep expires > > You might also try installing the > freeipa-healthcheck package and > running ipa-healthcheck. Expect a lot of > errors since it won't be able > to connect to the CA but it will also check > the validity dates, etc. > > > ipa2: > > Fedora 35 > > freeipa-server-4.9.11-1.fc35.x86_64 > > Managed suffixes: domain > > > > So my thought process was: if it can not > authenticate against the CA > > REST API, I need to add the CA capability > to ipa2 > > You need to authenticate to the CA to create > a clone of it. You can't > install another CA until you get your > existing one working. > > rob > > > -- > _______________________________________________ > FreeIPA-users mailing list -- > [email protected] > <mailto:[email protected]> > To unsubscribe send an email to > [email protected] > <mailto:[email protected]> > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > > https://lists.fedorahosted.org/archives/list/[email protected] > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > > > > -- > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich > diesmal abweichend im groüen Saal. > > > > -- > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal > abweichend im groüen Saal. > > > > -- > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im > groüen Saal. -- _______________________________________________ FreeIPA-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
