Re: [Freeipa-users] I think I lost my CA...
Oops, the slapd messages are arriving every 60s, not 5m. On 05/18/2017 08:56 AM, Bret Wortman wrote: httpd_error seems to give the most information. When i try to use ipa cert-show: ipa: INFO: [jsonserver_kerb] ad...@damascusgrp.com: ping(): SUCCESS (111)Connection refused: AH00957: AJP: attempt to connect to 127.0.0.1:8009 (localhost) failed AH00959: ap_proxy_connect_backend disabling worker for (locahost) for 60s [client 192.168.208.54:52714] AH00896: failed to make connection to backend: localhost ipa: ERROR: ra.get_certificate(): Unable to communicate with CMS (503) ipa: INFO: [jsonserver_kerb] ad...@damascusgrp.com: cert_show/1(u'895', version=u'2.213'): CertificateOperationError /var/log/pki/pki-tomcat/ca/debug just loops through the same set of messages every 5 minutes or so but doesn't seem to error. /var/log/pki/localhost_access_log.2017-05-18.txt is basically empty except for a single entry (for a POST to /ca/admin/ca/getStatus) Nothing shows up in dirsrv/slapd-DAMASCUSGRP-COM/errors or access when I issue the request, but periodic messages do appear about every 5 minutes or so. On 05/18/2017 08:43 AM, Bret Wortman wrote: On 04/26/2017 06:02 PM, Rob Crittenden wrote: Bret Wortman wrote: So I can see my certs using cert-find, but can't get details using cert-show or add new ones using cert-request. # ipa cert-find : -- Number of entries returned 385 -- # ipa cert-show 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-show 1 (which does not exist) ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-status 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # Is this an IPV6 thing? Because ipactl shows everything green and certmonger is running. Doubtful. cert-find and cert-show use different APIs in dogtag. cert-find uses the newer RESTful API and cert-show uses the older XML-based API (and is authenticated). I'm guessing that is where the issue lies. What I'd recommend doing is noting the time, restarting the CA, and then plow through the debug log looking for failures. It could be that the CA is only partially up (and I'd check your CA subsystem certs as well). Which debug log, specifically, do you think will help? I'm also not sure what you mean by, "check your CA subsystem certs." We still have pending CSRs that we can't grant until I get this working again. rob Bret On 04/26/2017 09:03 AM, Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg -
Re: [Freeipa-users] I think I lost my CA...
httpd_error seems to give the most information. When i try to use ipa cert-show: ipa: INFO: [jsonserver_kerb] ad...@damascusgrp.com: ping(): SUCCESS (111)Connection refused: AH00957: AJP: attempt to connect to 127.0.0.1:8009 (localhost) failed AH00959: ap_proxy_connect_backend disabling worker for (locahost) for 60s [client 192.168.208.54:52714] AH00896: failed to make connection to backend: localhost ipa: ERROR: ra.get_certificate(): Unable to communicate with CMS (503) ipa: INFO: [jsonserver_kerb] ad...@damascusgrp.com: cert_show/1(u'895', version=u'2.213'): CertificateOperationError /var/log/pki/pki-tomcat/ca/debug just loops through the same set of messages every 5 minutes or so but doesn't seem to error. /var/log/pki/localhost_access_log.2017-05-18.txt is basically empty except for a single entry (for a POST to /ca/admin/ca/getStatus) Nothing shows up in dirsrv/slapd-DAMASCUSGRP-COM/errors or access when I issue the request, but periodic messages do appear about every 5 minutes or so. On 05/18/2017 08:43 AM, Bret Wortman wrote: On 04/26/2017 06:02 PM, Rob Crittenden wrote: Bret Wortman wrote: So I can see my certs using cert-find, but can't get details using cert-show or add new ones using cert-request. # ipa cert-find : -- Number of entries returned 385 -- # ipa cert-show 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-show 1 (which does not exist) ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-status 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # Is this an IPV6 thing? Because ipactl shows everything green and certmonger is running. Doubtful. cert-find and cert-show use different APIs in dogtag. cert-find uses the newer RESTful API and cert-show uses the older XML-based API (and is authenticated). I'm guessing that is where the issue lies. What I'd recommend doing is noting the time, restarting the CA, and then plow through the debug log looking for failures. It could be that the CA is only partially up (and I'd check your CA subsystem certs as well). Which debug log, specifically, do you think will help? I'm also not sure what you mean by, "check your CA subsystem certs." We still have pending CSRs that we can't grant until I get this working again. rob Bret On 04/26/2017 09:03 AM, Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST
Re: [Freeipa-users] I think I lost my CA...
On 04/26/2017 06:02 PM, Rob Crittenden wrote:
Bret Wortman wrote:
So I can see my certs using cert-find, but can't get details using
cert-show or add new ones using cert-request.
# ipa cert-find
:
--
Number of entries returned 385
--
# ipa cert-show 895
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
# ipa cert-show 1 (which does not exist)
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
# ipa cert-status 895
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
#
Is this an IPV6 thing? Because ipactl shows everything green and
certmonger is running.
Doubtful.
cert-find and cert-show use different APIs in dogtag. cert-find uses the
newer RESTful API and cert-show uses the older XML-based API (and is
authenticated). I'm guessing that is where the issue lies.
What I'd recommend doing is noting the time, restarting the CA, and then
plow through the debug log looking for failures. It could be that the CA
is only partially up (and I'd check your CA subsystem certs as well).
Which debug log, specifically, do you think will help? I'm also not sure
what you mean by, "check your CA subsystem certs." We still have pending
CSRs that we can't grant until I get this working again.
rob
Bret
On 04/26/2017 09:03 AM, Bret Wortman wrote:
Digging still deeper:
# ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
Looks like this is an HTTP error; so is it possible that my IPA thinks
it has a CA but there's no CMS available?
On 04/26/2017 08:41 AM, Bret Wortman wrote:
Using the firefox debugger, I get these errors when trying to pop up
the New Certificate dialog:
Empty string passed to getElementById(). (5)
jquery.js:4:1060
TypeError: u is undefined
app.js:1:362059
Empty string passed to getElementById(). (5)
jquery.js:4:1060
TypeError: t is undefined
app.js:1:217432
I'm definitely not a web kind of guy so I'm not sure if this is
helpful or not. This is on 4.4.0, API Version 2.213.
Bret
On 04/26/2017 08:35 AM, Bret Wortman wrote:
Good news. One of my servers _does_ have CA installed. So why does
"Action -> New Certificate" not do anything on this or any other server?
Bret
On 04/25/2017 02:52 PM, Bret Wortman wrote:
I recently had to upgrade all my Fedora IPA servers to C7. It went
well, and we've been up and running nicely on 4.4.0 on C7 for the
past month or so.
Today, someone came and asked me to generate a new certificate for
their web server. All was good until I went to the IPA UI and tried
to perform Actions->New Certificate, which did nothing. I tried
each of our 3 servers in turn. All came back with no popup window
and no error, either.
I suspect the problem might be that we no longer have a CA server
due to the method I used to upgrade the servers. I likely missed a
"--setup-ca" in there somewhere, so my rolling update rolled over
the CA.
What's my best hope of recovery? I never ran this before, so I'm
not sure if this shows that I'm missing a CA or not:
# ipa ca-find
1 CA matched
Name: ipa
Description IPA CA
Authority ID: 3ce3346[...]
Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM
Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM
Number of entries returned 1
# ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA,
O=DAMASCUSGRP.COM"
ipa: ERROR: Failed to authenticate to CA REST API
# klist
Ticket cache: KEYRING:persistent:0:0
Default principal: ad...@damascusgrp.com
Valid starting Expires Service principal
04/25/2017 18:48:26 04/26/2017 18:48:21
krbtgt/damascusgrp@damascusgrp.com
#
What's my best path of recovery?
--
*Bret Wortman*
The Damascus Group
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
The log slog continues but isn't turning up anything useful, or I'm looking in the wrong logs. Now getting twice-daily visits from users who need new SSL certs wondering when I'm going to be able to create them. I'm happy to do the work to figure out what went wrong, I just don't grok these individual components at this level very well. When something goes wrong, it's not trivial to solve. Well, for me it isn't, anyway. ;-) Bret On 05/02/2017 10:50 AM, Bret Wortman wrote: I plowed through /var/log/pki/pki-tomcat/ca/debug, but nothing jumps out as looking like an error. The cert-show failure is troubling, but my inability to get CSRs turned into certs is what's actually driving this. Bret On 04/26/2017 06:02 PM, Rob Crittenden wrote: Bret Wortman wrote: So I can see my certs using cert-find, but can't get details using cert-show or add new ones using cert-request. # ipa cert-find : -- Number of entries returned 385 -- # ipa cert-show 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-show 1 (which does not exist) ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-status 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # Is this an IPV6 thing? Because ipactl shows everything green and certmonger is running. Doubtful. cert-find and cert-show use different APIs in dogtag. cert-find uses the newer RESTful API and cert-show uses the older XML-based API (and is authenticated). I'm guessing that is where the issue lies. What I'd recommend doing is noting the time, restarting the CA, and then plow through the debug log looking for failures. It could be that the CA is only partially up (and I'd check your CA subsystem certs as well). rob Bret On 04/26/2017 09:03 AM, Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Fwd: dirsrv not starting after unplanned outage
That was it. Minor edits (nsslapd-localhost) and we're up and running. Thanks, Ludwig! On 05/09/2017 06:50 AM, Ludwig Krispenz wrote: looks like you lost your configuration files dse.ldif and its backup as well during the outage. could you check what you have in /etc/dirsrv/slapd- you can try to copy one of the *dse.ldif* to dse.ldif and try to restart, but that file maybe up to date. Ludwig On 05/09/2017 12:00 PM, Bret Wortman wrote: We had an unplanned power outage which may have affected one of our freeipa servers. When trying to start, it now errors out. # ipactl start Starting Directory Service Failed to start Directory Service: Command '/bin/systemctl start dirsrv@SPX-NET.service' returned non-zero exit status 1 # In /var/log/messages, there is a lengthy list of errors like this: 2017-05-09T09:25:40.178252+00:00 asipa ns-slapd: [09/May/2017:09:25:40.159091115 +] valueset_value_symtax_cmp: slapi_attr_bvalues2keys_sv failed for type attributetypes ending with: 2017-05-09T09:25:40.178438+00:00 asiopa ns-slapd: [09/May/2017:09:25:40:161987520 +] dse_read_one_file - The entry cn=schema in file /etc/dirsrv/slapd-DAMASCUSGRP.COM /schema/00core.ldif (lineno: 1) is invalid, error code 21 (Invalid syntax) - attribute type aci: Unknown attribute syntax OID "1.3.6.1.4.1.1466.115.121.1.15" 2017-05-09T09:25:40.178469+00:00 asipa ns-slapd: [09/May/2017:09:25:40.162014035 +] dse - Please edit the file to correct the reported problems and then restart the server. The entry in 00core.ldif was verified against the other servers and the file hasn't been altered or edited that I can see. Where else can I look? I've got two servers up, but I'd like to have all 3 operational. -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Fwd: dirsrv not starting after unplanned outage
We had an unplanned power outage which may have affected one of our freeipa servers. When trying to start, it now errors out. # ipactl start Starting Directory Service Failed to start Directory Service: Command '/bin/systemctl start dirsrv@SPX-NET.service' returned non-zero exit status 1 # In /var/log/messages, there is a lengthy list of errors like this: 2017-05-09T09:25:40.178252+00:00 asipa ns-slapd: [09/May/2017:09:25:40.159091115 +] valueset_value_symtax_cmp: slapi_attr_bvalues2keys_sv failed for type attributetypes ending with: 2017-05-09T09:25:40.178438+00:00 asiopa ns-slapd: [09/May/2017:09:25:40:161987520 +] dse_read_one_file - The entry cn=schema in file /etc/dirsrv/slapd-DAMASCUSGRP.COM /schema/00core.ldif (lineno: 1) is invalid, error code 21 (Invalid syntax) - attribute type aci: Unknown attribute syntax OID "1.3.6.1.4.1.1466.115.121.1.15" 2017-05-09T09:25:40.178469+00:00 asipa ns-slapd: [09/May/2017:09:25:40.162014035 +] dse - Please edit the file to correct the reported problems and then restart the server. The entry in 00core.ldif was verified against the other servers and the file hasn't been altered or edited that I can see. Where else can I look? I've got two servers up, but I'd like to have all 3 operational. -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
The closest I found was this: [02/May/2017:14:33:57][localhost-startStop-1]: No rule can be found for publishing: cacert [02/May/2017:14:33:37][localhost-startStop-1]: published ca cert [02/May/2017:14:33:37][localhost-startStop-1]: CMSEngine: ca startup done On 05/02/2017 10:50 AM, Bret Wortman wrote: I plowed through /var/log/pki/pki-tomcat/ca/debug, but nothing jumps out as looking like an error. The cert-show failure is troubling, but my inability to get CSRs turned into certs is what's actually driving this. Bret On 04/26/2017 06:02 PM, Rob Crittenden wrote: Bret Wortman wrote: So I can see my certs using cert-find, but can't get details using cert-show or add new ones using cert-request. # ipa cert-find : -- Number of entries returned 385 -- # ipa cert-show 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-show 1 (which does not exist) ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-status 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # Is this an IPV6 thing? Because ipactl shows everything green and certmonger is running. Doubtful. cert-find and cert-show use different APIs in dogtag. cert-find uses the newer RESTful API and cert-show uses the older XML-based API (and is authenticated). I'm guessing that is where the issue lies. What I'd recommend doing is noting the time, restarting the CA, and then plow through the debug log looking for failures. It could be that the CA is only partially up (and I'd check your CA subsystem certs as well). rob Bret On 04/26/2017 09:03 AM, Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
I plowed through /var/log/pki/pki-tomcat/ca/debug, but nothing jumps out as looking like an error. The cert-show failure is troubling, but my inability to get CSRs turned into certs is what's actually driving this. Bret On 04/26/2017 06:02 PM, Rob Crittenden wrote: Bret Wortman wrote: So I can see my certs using cert-find, but can't get details using cert-show or add new ones using cert-request. # ipa cert-find : -- Number of entries returned 385 -- # ipa cert-show 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-show 1 (which does not exist) ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-status 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # Is this an IPV6 thing? Because ipactl shows everything green and certmonger is running. Doubtful. cert-find and cert-show use different APIs in dogtag. cert-find uses the newer RESTful API and cert-show uses the older XML-based API (and is authenticated). I'm guessing that is where the issue lies. What I'd recommend doing is noting the time, restarting the CA, and then plow through the debug log looking for failures. It could be that the CA is only partially up (and I'd check your CA subsystem certs as well). rob Bret On 04/26/2017 09:03 AM, Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
Flo,
I did find that issue and made those corrections to our /etc/hosts file,
but the problem persists.
Thanks for the idea!
Bret
On 04/27/2017 03:42 AM, Florence Blanc-Renaud wrote:
On 04/26/2017 04:33 PM, Bret Wortman wrote:
So I can see my certs using cert-find, but can't get details using
cert-show or add new ones using cert-request.
# ipa cert-find
:
--
Number of entries returned 385
--
# ipa cert-show 895
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
# ipa cert-show 1 (which does not exist)
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
# ipa cert-status 895
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
#
Is this an IPV6 thing? Because ipactl shows everything green and
certmonger is running.
Hi Bret,
the issue looks similar to https://pagure.io/freeipa/issue/6575 and
https://pagure.io/dogtagpki/issue/2570 which were IPv6 related. Note
that IPv6 must be enabled on the machine but IPA does not require an
IPv6 address to be configured (except for the loopback).
You can check the following:
- is PKI listening to port 8009 on IPv6 or IPv4 interface?
sudo netstat -tunpl | grep 8009
tcp6 0 0 127.0.0.1:8009 :::* LISTEN 10749/java
- /etc/pki/pki-tomcat/server.xml defines a redirection from port 8009
to 8443, and the "address" part is important:
In the above example, it will be using localhost which can resolve
either to IPv4 or IPv6.
- /etc/hosts must define the loopback addresses with
127.0.0.1 localhost localhost.localdomain localhost4
localhost4.localdomain4
::1 localhost localhost.localdomain localhost6
localhost6.localdomain6
HTH,
Flo.
Bret
On 04/26/2017 09:03 AM, Bret Wortman wrote:
Digging still deeper:
# ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (503)
Looks like this is an HTTP error; so is it possible that my IPA thinks
it has a CA but there's no CMS available?
On 04/26/2017 08:41 AM, Bret Wortman wrote:
Using the firefox debugger, I get these errors when trying to pop up
the New Certificate dialog:
Empty string passed to getElementById(). (5)
jquery.js:4:1060
TypeError: u is undefined
app.js:1:362059
Empty string passed to getElementById(). (5)
jquery.js:4:1060
TypeError: t is undefined
app.js:1:217432
I'm definitely not a web kind of guy so I'm not sure if this is
helpful or not. This is on 4.4.0, API Version 2.213.
Bret
On 04/26/2017 08:35 AM, Bret Wortman wrote:
Good news. One of my servers _does_ have CA installed. So why does
"Action -> New Certificate" not do anything on this or any other
server?
Bret
On 04/25/2017 02:52 PM, Bret Wortman wrote:
I recently had to upgrade all my Fedora IPA servers to C7. It went
well, and we've been up and running nicely on 4.4.0 on C7 for the
past month or so.
Today, someone came and asked me to generate a new certificate for
their web server. All was good until I went to the IPA UI and tried
to perform Actions->New Certificate, which did nothing. I tried
each of our 3 servers in turn. All came back with no popup window
and no error, either.
I suspect the problem might be that we no longer have a CA server
due to the method I used to upgrade the servers. I likely missed a
"--setup-ca" in there somewhere, so my rolling update rolled over
the CA.
What's my best hope of recovery? I never ran this before, so I'm
not sure if this shows that I'm missing a CA or not:
# ipa ca-find
1 CA matched
Name: ipa
Description IPA CA
Authority ID: 3ce3346[...]
Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM
Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM
Number of entries returned 1
# ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA,
O=DAMASCUSGRP.COM"
ipa: ERROR: Failed to authenticate to CA REST API
# klist
Ticket cache: KEYRING:persistent:0:0
Default principal: ad...@damascusgrp.com
Valid starting Expires Service principal
04/25/2017 18:48:26 04/26/2017 18:48:21
krbtgt/damascusgrp@damascusgrp.com
#
What's my best path of recovery?
--
*Bret Wortman*
The Damascus Group
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
On 04/26/2017 10:22 AM, Rob Crittenden wrote: Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? Apache proxies requests to the CA so there could be a mismatch I suppose. I'd ensure that the pki processes are running on the box for starters and then dig into the CA debug log for more details. Is that /var/log/pki/pki-tomcat/ca/debug? If so, then nothing happens in it during the above operations. As you noted, apache produces the following when trying to show a valid cert even though there's nothing in what I think is the pki ca debug log. ps aux shows pki processes alive, at least, and in ownership of the 8009 port (verified by lsof). [Wed Apr 26 14:38:48.157961 2017] [:error] [pid 15801] ipa: INFO: [jsonserver_session] ad...@damascusgrp.com: ping(): SUCCESS [Wed Apr 26 14:38:48.247040 2017] [proxy:error] [pid 15804] (111)Connection refused: AH00957: AJP: attempt to connect to 127.0.0.1:8009 (localhost) failed [Wed Apr 26 14:38:48.247072 2017] [proxy:error] [pid 15804] AH00959: ap_proxy_connect_)backend disabling worker for (localhost) for 60s [Wed Apr 26 14:38:48.247078 2017] [proxy_ajp:error] [pid 15804] [client 192.168.208.54:56618] AH00896: failed to make connection to backend: localhost [Wed Apr 26 14:38:48.247531 2017] [:error] [pid 15800] ipa: ERROR: ra.get_certificate(): Unable to communicate with CMS (503) [Wed Apr 26 14:38:48.247765 2017] [:error] [pid 15800] ipa: INFO: [jsonserver_session] ad...@damascusgrp.com: cert_show/1(u'895', version=u'2.213'): CertificateOperationError rob On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
So I can see my certs using cert-find, but can't get details using cert-show or add new ones using cert-request. # ipa cert-find : -- Number of entries returned 385 -- # ipa cert-show 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-show 1 (which does not exist) ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # ipa cert-status 895 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) # Is this an IPV6 thing? Because ipactl shows everything green and certmonger is running. Bret On 04/26/2017 09:03 AM, Bret Wortman wrote: Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
Digging still deeper: # ipa cert-request f.f --principal=HTTP/`hostname`@DAMASCUSGRP.COM ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) Looks like this is an HTTP error; so is it possible that my IPA thinks it has a CA but there's no CMS available? On 04/26/2017 08:41 AM, Bret Wortman wrote: Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
Using the firefox debugger, I get these errors when trying to pop up the New Certificate dialog: Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: u is undefined app.js:1:362059 Empty string passed to getElementById(). (5) jquery.js:4:1060 TypeError: t is undefined app.js:1:217432 I'm definitely not a web kind of guy so I'm not sure if this is helpful or not. This is on 4.4.0, API Version 2.213. Bret On 04/26/2017 08:35 AM, Bret Wortman wrote: Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] I think I lost my CA...
Good news. One of my servers _does_ have CA installed. So why does "Action -> New Certificate" not do anything on this or any other server? Bret On 04/25/2017 02:52 PM, Bret Wortman wrote: I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] I think I lost my CA...
I recently had to upgrade all my Fedora IPA servers to C7. It went well, and we've been up and running nicely on 4.4.0 on C7 for the past month or so. Today, someone came and asked me to generate a new certificate for their web server. All was good until I went to the IPA UI and tried to perform Actions->New Certificate, which did nothing. I tried each of our 3 servers in turn. All came back with no popup window and no error, either. I suspect the problem might be that we no longer have a CA server due to the method I used to upgrade the servers. I likely missed a "--setup-ca" in there somewhere, so my rolling update rolled over the CA. What's my best hope of recovery? I never ran this before, so I'm not sure if this shows that I'm missing a CA or not: # ipa ca-find 1 CA matched Name: ipa Description IPA CA Authority ID: 3ce3346[...] Subject DN: CN=Certificate Authority, O=DAMASCUSGRP.COM Issuer DN: CN=Certificate Authority,O=DAMASCUSGRP.COM Number of entries returned 1 # ipa ca-add dg --desc "Damascus Group" --subject "CN=DG CA, O=DAMASCUSGRP.COM" ipa: ERROR: Failed to authenticate to CA REST API # klist Ticket cache: KEYRING:persistent:0:0 Default principal: ad...@damascusgrp.com Valid starting Expires Service principal 04/25/2017 18:48:26 04/26/2017 18:48:21 krbtgt/damascusgrp@damascusgrp.com # What's my best path of recovery? -- *Bret Wortman* The Damascus Group -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] FreeIPA update guidance
I don't know that what we did is the most correct or even best way to manage an upgrade, but here's what I did. We started with two nodes, ipa1 and ipa2. Both running Fedora. I built a new system, ipa3, and installed IPA on it, then made it a replica. I then removed the replication agreements to ipa1 and upgraded it. Then made it a replica again using ipa3 as the master. Finally, I removed ipa2's replication agreement and upgraded it. Again, it was brought back into replication by creating a replication file on ipa3 and copying it to ipa2. Somewhere in there, I'm pretty sure I had to do something with the CA to ensure we still had one, but for the life of me, I can't remember what I did! Good luck! Bret On 04/21/2017 10:06 AM, B.harries wrote: Hi All, As I am new to the list, I'd like to introduce myself as Bennie. In my fairly small (CentOS based) organization we use FreeIPA and we are honestly really happy with this all in one solution. Lately however we are facing an issue regarding updating FreeIPA and I was hoping I could find some guidance on this mail list =). *Current situation* We are currently running FreeIPA 4.3.1 on Fedora 23. When we started using FreeIPA, CentOS was lacking quite behind so we choose to go with Fedora. As Fedora 23 is quite out of date now we tried to perform a dist-upgrade, enabling us to continue using FreeIPA on the 4.4 branch. This dist-upgrade however led to an inoperable condition of FreeIPA, mainly the PKI service fails miserably. *Second attempt* We then tried to install a fresh CentOS server, having FreeIPA version 4.4 and attaching it as a second master to our IPA instance. This however didn't work out as well, probably because the directory structures are not equal. So far, everything failed. I was wondering if anyone here faced similar problems and might be able to point in the right direction? Thanks in advance for a reply! Bennie -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migrate IPA cluster F21 -> C7
I saw as I was working through it, and it's in fact what I did. Migrating the last server to CentOS right now. Thanks for the help! On 03/29/2017 09:53 AM, Rob Crittenden wrote: Bret Wortman wrote: Never mind. Lost my mind. ipa-replica-install followed by ipa-ca-install appears to be the ticket. Or you can do it in one step by passing --setup-ca to ipa-replica-install rob Bret On 03/29/2017 06:22 AM, Bret Wortman wrote: I've tried googling but keep coming up with beer recipes. How do you suggest adding the replica CA? I'm piecing together the options I want on my ipa-server-install command and am trying to understand the CA-related options. Thanks! Bret On 03/28/2017 08:45 AM, Bret Wortman wrote: I'm studying the best way to migrate out IPA servers (there are two) from F21 to C7. I _think_ the sequence of steps I need to perform is: 1. Build new C7 IPA server (ipa-c) and enable replication to it. 2. Migrate CA functions from our existing CA server (ipa-a) to this new one (ipa-c). 3. Upgrade ipa-b to C7 and enable replication to it. 4. Either upgrade ipa-a and have a third ipa server or discard the vm in favor of the two now in service. Am I missing anything? Making this harder than it needs to be? Our F21 servers are using IPA 4.1.4-1 (and pki-ca 10.2.1-3) so I'm not if replication across versions is supported between these and IPA 4.4.0 (pki-ca 10.3.3). -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migrate IPA cluster F21 -> C7
Never mind. Lost my mind. ipa-replica-install followed by ipa-ca-install appears to be the ticket. Bret On 03/29/2017 06:22 AM, Bret Wortman wrote: I've tried googling but keep coming up with beer recipes. How do you suggest adding the replica CA? I'm piecing together the options I want on my ipa-server-install command and am trying to understand the CA-related options. Thanks! Bret On 03/28/2017 08:45 AM, Bret Wortman wrote: I'm studying the best way to migrate out IPA servers (there are two) from F21 to C7. I _think_ the sequence of steps I need to perform is: 1. Build new C7 IPA server (ipa-c) and enable replication to it. 2. Migrate CA functions from our existing CA server (ipa-a) to this new one (ipa-c). 3. Upgrade ipa-b to C7 and enable replication to it. 4. Either upgrade ipa-a and have a third ipa server or discard the vm in favor of the two now in service. Am I missing anything? Making this harder than it needs to be? Our F21 servers are using IPA 4.1.4-1 (and pki-ca 10.2.1-3) so I'm not if replication across versions is supported between these and IPA 4.4.0 (pki-ca 10.3.3). -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migrate IPA cluster F21 -> C7
I've tried googling but keep coming up with beer recipes. How do you suggest adding the replica CA? I'm piecing together the options I want on my ipa-server-install command and am trying to understand the CA-related options. Thanks! Bret On 03/28/2017 08:45 AM, Bret Wortman wrote: I'm studying the best way to migrate out IPA servers (there are two) from F21 to C7. I _think_ the sequence of steps I need to perform is: 1. Build new C7 IPA server (ipa-c) and enable replication to it. 2. Migrate CA functions from our existing CA server (ipa-a) to this new one (ipa-c). 3. Upgrade ipa-b to C7 and enable replication to it. 4. Either upgrade ipa-a and have a third ipa server or discard the vm in favor of the two now in service. Am I missing anything? Making this harder than it needs to be? Our F21 servers are using IPA 4.1.4-1 (and pki-ca 10.2.1-3) so I'm not if replication across versions is supported between these and IPA 4.4.0 (pki-ca 10.3.3). -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Migrate IPA cluster F21 -> C7
I'm studying the best way to migrate out IPA servers (there are two) from F21 to C7. I _think_ the sequence of steps I need to perform is: 1. Build new C7 IPA server (ipa-c) and enable replication to it. 2. Migrate CA functions from our existing CA server (ipa-a) to this new one (ipa-c). 3. Upgrade ipa-b to C7 and enable replication to it. 4. Either upgrade ipa-a and have a third ipa server or discard the vm in favor of the two now in service. Am I missing anything? Making this harder than it needs to be? Our F21 servers are using IPA 4.1.4-1 (and pki-ca 10.2.1-3) so I'm not if replication across versions is supported between these and IPA 4.4.0 (pki-ca 10.3.3). -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Best way to upgrade IPA servers from Fedora
Our IPA infrastructure is based entirely on servers running Fedora 21. We need (desperately) to upgrade these to C7 or RHEL. All are currently VMs. Does it make sense to: 1. Add a new host already running C7 to increase our total servers from 3 to 4 (call it IPA4) 2. Remove IPA1 and upgrade it to C7, install the IPA server packages and join it back into the collective 3. Remove IPA2 and upgrade it like IPA1. Add it back. 4. Somehow migrate the CA function to one of the C7 nodes from IPA3. Trash or repurpose this VM. What's the right way to do this? If I've got it right here, what's the best way to move the CA function from the node it's on now to one of the freshly-upgraded hosts? Thanks! -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Signing certs with longer lifetimes (FreeIPA CA)
I'm generating CSRs like this:
# certutil -R -d $DB -a -g 2048 -v 60 -s "CN=${HOST},O=DAMASCUSGRP.COM" -8
${SHORTHOST},${HOST}
Then pasting this into the web interface of our IPA instance under
"Actions->New Certificate" on the host's page. I then use Actions->View
Certificate and see that it expires in 2019.
I want that cert to expire in 2022. What do I need to change to make
that happen, and what's the right way to do it? I looked at some of the
scripts & files under /etc/pki and see references to $DAYS that look to
do what I want, but I don't want to do something that'll get clobbered
at the next IPA upgrade.
Bret
On 01/19/2017 10:30 AM, Kimi Rachel wrote:
Mail
heyy Bret, how are you? lets talk details ..
On Thu, Jan 19, 2017 at 9:30 PM, Bret Wortman
<bret.wort...@damascusgrp.com <mailto:bret.wort...@damascusgrp.com>>
wrote:
It seems all our certs being signed by the FreeIPA CA are given 2
year expirations. We'd like to increase that to 5 years. I've
added "-v 60" to our certutil commands generating the CSRs, but
the CA is still only issuing 24 month certs.
What do I need to change to issue certs with longer lifetimes? We
really don't want to go around every 2 years and reissue certs...
--
*Bret Wortman*
Damascus Products
ph/fax: 1-855-644-2783
Wrap Buddies InDemand at
http://bwortman.us/2ieQN4t
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
[Freeipa-users] Signing certs with longer lifetimes (FreeIPA CA)
It seems all our certs being signed by the FreeIPA CA are given 2 year expirations. We'd like to increase that to 5 years. I've added "-v 60" to our certutil commands generating the CSRs, but the CA is still only issuing 24 month certs. What do I need to change to issue certs with longer lifetimes? We really don't want to go around every 2 years and reissue certs... -- *Bret Wortman* Damascus Products ph/fax: 1-855-644-2783 Wrap Buddies InDemand at http://bwortman.us/2ieQN4t -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] How to get a new cert
Perfect. That did the trick. Many thanks, Flo. Bret On 09/28/2016 09:47 AM, Florence Blanc-Renaud wrote: On 09/27/2016 08:00 PM, Bret Wortman wrote: That looks like it worked, but I have a follow-on question: I need to provide my RabbitMQ instance with a cacert file, a cert, and a key file. These seem to be .pem files. Is there an easy way to gather these 3 files from a typical IPA client node? Hi, you can retrieve the new cert using the GUI. Navigate to Identity tab, then Users or Hosts or Services and pick your user, host or service. You will find in the "Actions" button a command to "Get Certificate". This will open a new window with the content of the cert, that you can copy/paste into mycert.pem. Once you have obtained mycert.pem, you can add it to the NSS database that you used previously in order to generate the CSR: $ certutil -A -d path_to_database -i mycert.pem -t u,u,u -n mycert Add IPA CA to the nss database: $ certutil -A -d path_to_database -n "IPA CA" -t CT,, -a < /etc/ipa/ca.crt Then pk12util and openssl will allow you to extract the key and certs through a temp keys.p12 file: $ pk12util -o keys.p12 -n mycert -d path_to_database $ openssl pkcs12 -in keys.p12 -out mykey.pem -nodes The output is mykey.pem which contains the key, the new certificate and IPA CA certificate. HTH, Flo. Merci! Bret On 09/27/2016 11:28 AM, Florence Blanc-Renaud wrote: Hi Bret, would the following be helpful? In "Linux Domain Identity, Authentication, and Policy Guide", Chapter 17.1.1 Requesting New Certificates for a User, Host, or Service [1] Flo. [1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/certificates.html#certificate-request On 09/27/2016 04:20 PM, Bret Wortman wrote: Is there a guide anywhere for how to obtain an SSL certificate for a new server & service from the IPA CA master? Most of the guides I'm seeing online use web pages at the major CAs to do this and I'd like to keep it in the family. Thanks! -- *Bret Wortman* <http://wrapbuddies.co/> http://wrapbuddies.co/ -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] How to get a new cert
Yeah, I'm still not getting this, and I'm probably missing something painfully obvious. I follow the steps here: 1. Log into the server for which I need the cert. 2. # certutil -R -d /etc/pki/nssdb -a -g 2048 -s "CN=testesk1.internal.net,O=INTERNAL.NET" > ssl.csr I then copy the contents of the csr file and paste it into the FreeIPA UI after selecting Actions->New Certificiate from the Host Settings page. 3. I then click Actions->Get Certificate on that same page to extract the contents and paste it into a new .pem file on the requesting host. But how do I get at the key that was used in the creation of this cert? I can get the cacert, and I've got the newly-issued cert, but what about the key? Thanks! Bret On 09/27/2016 02:00 PM, Bret Wortman wrote: That looks like it worked, but I have a follow-on question: I need to provide my RabbitMQ instance with a cacert file, a cert, and a key file. These seem to be .pem files. Is there an easy way to gather these 3 files from a typical IPA client node? Merci! Bret On 09/27/2016 11:28 AM, Florence Blanc-Renaud wrote: Hi Bret, would the following be helpful? In "Linux Domain Identity, Authentication, and Policy Guide", Chapter 17.1.1 Requesting New Certificates for a User, Host, or Service [1] Flo. [1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/certificates.html#certificate-request On 09/27/2016 04:20 PM, Bret Wortman wrote: Is there a guide anywhere for how to obtain an SSL certificate for a new server & service from the IPA CA master? Most of the guides I'm seeing online use web pages at the major CAs to do this and I'd like to keep it in the family. Thanks! -- *Bret Wortman* <http://wrapbuddies.co/> http://wrapbuddies.co/ -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] How to get a new cert
That looks like it worked, but I have a follow-on question: I need to provide my RabbitMQ instance with a cacert file, a cert, and a key file. These seem to be .pem files. Is there an easy way to gather these 3 files from a typical IPA client node? Merci! Bret On 09/27/2016 11:28 AM, Florence Blanc-Renaud wrote: Hi Bret, would the following be helpful? In "Linux Domain Identity, Authentication, and Policy Guide", Chapter 17.1.1 Requesting New Certificates for a User, Host, or Service [1] Flo. [1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/certificates.html#certificate-request On 09/27/2016 04:20 PM, Bret Wortman wrote: Is there a guide anywhere for how to obtain an SSL certificate for a new server & service from the IPA CA master? Most of the guides I'm seeing online use web pages at the major CAs to do this and I'd like to keep it in the family. Thanks! -- *Bret Wortman* <http://wrapbuddies.co/> http://wrapbuddies.co/ -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] How to get a new cert
Is there a guide anywhere for how to obtain an SSL certificate for a new server & service from the IPA CA master? Most of the guides I'm seeing online use web pages at the major CAs to do this and I'd like to keep it in the family. Thanks! -- *Bret Wortman* <http://wrapbuddies.co/> http://wrapbuddies.co/ -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Using our IPA CA as a trusted CA to sign ssl certificates
On 06/03/2016 01:04 PM, Rob Crittenden wrote: Bret Wortman wrote: On 06/03/2016 11:02 AM, Rob Crittenden wrote: Bret Wortman wrote: I'm not sure I'd call what we have "success" just yet. ;-) You're right -- F21, IPA 4.1.4-1. I'll try the steps you outlined and see how we go. Rob, would you have just used the existing "localhost.key" instead of generating a new one? No, I think you did the right thing, the default keysize was probably still 1024 in F21. I double-checked the getcert-request man page and it looks like it will use an existing key if one exists in the key file passed in so I was wrong about that bit. You just didn't need to use req to generate a CSR as certmonger will do that for you. Good to know. I tried the update-ca-trust on both the yum server and on my workstation but nothing changed even after an httpd restart. I did take a peek inside /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt and didn't see my /etc/ipa/ca.crt in there (which may not be a problem, but I confess I'm not sure what should be where at this point). You'd only need to do this on the machine acting as a client. I'm pretty sure yum uses /etc/pki/nssdb. Is the IPA CA in there and trusted? $ certutil -L -d /etc/pki/nssdb It's in there on both the server and client. rob Bret rob On 06/03/2016 09:48 AM, Rob Crittenden wrote: Bret Wortman wrote: So for our internal yum server, I created a new key and cert request (it had a localhost key and cert but I wanted to start clean): # openssl genrsa 2048 > /etc/pki/tls/private/server.key # openssl req -new -x509 -nodes -sha1 -days 365 -key /etc/pki/tls/private/server.key > /etc/pki/tls/certs/server.crt # ipa-getcert request -f /etc/pki/tls/certs/server.crt -k /etc/pki/tls/private/server.key -r I try not to argue with success but I'd be curious what is actually going on here. You generate a CSR and call it a certificate. It is probably the case that certmonger is ignoring it altogether and generating its own CSR. ipa-getcert list shows it approved. I set up SSL in apache to use the above .key and .crt, but when I try to run yum against this using ssl: # yum search ffmpeg Loaded plugins: langpacks https://yum.private.net/fedora/releases/21/Everything/x86_64/os/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's certificate issuer has been marked as not trusted by the user." : Is there a step I need to take on the clients so they'll accept this cert as trusted? I thought having it be signed by the IPA CA would have taken care of that. # ls -l /etc/ipa/ca.crt -rw-r--r-- 1 root root 2546 Apr 28 2014 /etc/ipa/ca.crt # Pretty much only IPA tools know to use this file. My knowledge is a bit stale on adding the IPA CA to the global trust but I'm pretty sure it is done automatically now and I think it was in the 4.2 timeframe. I'm assuming this is Fedora 21 so it doesn't have this code. Look at this, https://fedoraproject.org/wiki/Features/SharedSystemCertificates The idea is to add the IPA CA to that and then all tools using SSL would "just work". Something like: # cp /etc/ipa/ca.crt /usr/share/pki/ca-trust-source/anchors/ipa-ca.pem # update-ca-trust You'd need to remember to manually undo this if you ever redo your IPA install (and get a new CA): # rm /etc/ipa/ca.crt /usr/share/pki/ca-trust-source/anchors/ipa-ca.pem # update-ca-trust Like I said, I'm pretty sure this is all automatic in some more recent versions of IPA. rob --- Bret On 06/02/2016 07:25 PM, bret.wort...@damascusgrp.com wrote: Cool. I'll give this a go in the morning. Bret Wortman http://wrapbuddies.co/ On Jun 2, 2016, 6:24 PM -0400, Fraser Tweedale <ftwee...@redhat.com>, wrote: On Thu, Jun 02, 2016 at 05:35:01PM -0400, bret.wort...@damascusgrp.com wrote: Sorry, let me back up a step. We need to implement hype everywhere. All our web services. And clients need to get keys automatically whether through IPA or Puppet. These systems use IPA for everything but authentication (to keep most users off). I'm trying to wuss out the easiest way to make this happen smoothly. Hi Bret, You can use the IPA CA to sign service certificates. See http://www.freeipa.org/page/Certmonger#Request_a_new_certificate. IPA-enrolled machines already have the IPA certificate in their trust store. If the clients are IPA-enrolled, everything should Just Work, otherwise you can distribute the IPA CA certificate to clients via Puppet** or whatever means you prefer. ** you will have to work out how, because I do not know Puppet :) Cheers, Fraser On Jun 2, 2016, 5:31 PM -0400, Rob Crittenden<rcrit...@redhat.com>, wrote: Bret Wortman wrote: Is it possible to use our freeipa CA as a trusted CA to sign our internal SSL certificates? Our system runs on a private network and so using the usual trusted sources isn't an option. We've been using self-
Re: [Freeipa-users] Using our IPA CA as a trusted CA to sign ssl certificates
I'll check and report back Tuesday. Bret Wortman http://wrapbuddies.co/ On Jun 3, 2016, 1:04 PM -0400, Rob Crittenden<rcrit...@redhat.com>, wrote: > Bret Wortman wrote: > > > > > > On 06/03/2016 11:02 AM, Rob Crittenden wrote: > > > Bret Wortman wrote: > > > > I'm not sure I'd call what we have "success" just yet. ;-) > > > > > > > > You're right -- F21, IPA 4.1.4-1. I'll try the steps you outlined and > > > > see how we go. > > > > > > > > Rob, would you have just used the existing "localhost.key" instead of > > > > generating a new one? > > > > > > No, I think you did the right thing, the default keysize was probably > > > still 1024 in F21. I double-checked the getcert-request man page and > > > it looks like it will use an existing key if one exists in the key > > > file passed in so I was wrong about that bit. You just didn't need to > > > use req to generate a CSR as certmonger will do that for you. > > > > > Good to know. > > > > I tried the update-ca-trust on both the yum server and on my workstation > > but nothing changed even after an httpd restart. I did take a peek > > inside /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt and > > didn't see my /etc/ipa/ca.crt in there (which may not be a problem, but > > I confess I'm not sure what should be where at this point). > > You'd only need to do this on the machine acting as a client. > > I'm pretty sure yum uses /etc/pki/nssdb. Is the IPA CA in there and trusted? > > $ certutil -L -d /etc/pki/nssdb > > rob > > > > > > > Bret > > > > > rob > > > > > > > > > > > > > > > On 06/03/2016 09:48 AM, Rob Crittenden wrote: > > > > > Bret Wortman wrote: > > > > > > So for our internal yum server, I created a new key and cert > > > > > > request (it > > > > > > had a localhost key and cert but I wanted to start clean): > > > > > > > > > > > > # openssl genrsa 2048>/etc/pki/tls/private/server.key > > > > > > # openssl req -new -x509 -nodes -sha1 -days 365 -key > > > > > > /etc/pki/tls/private/server.key>/etc/pki/tls/certs/server.crt > > > > > > # ipa-getcert request -f /etc/pki/tls/certs/server.crt -k > > > > > > /etc/pki/tls/private/server.key -r > > > > > > > > > > I try not to argue with success but I'd be curious what is actually > > > > > going on here. You generate a CSR and call it a certificate. It is > > > > > probably the case that certmonger is ignoring it altogether and > > > > > generating its own CSR. > > > > > > > > > > > ipa-getcert list shows it approved. I set up SSL in apache to use > > > > > > the > > > > > > above .key and .crt, but when I try to run yum against this using > > > > > > ssl: > > > > > > > > > > > > # yum search ffmpeg > > > > > > Loaded plugins: langpacks > > > > > > https://yum.private.net/fedora/releases/21/Everything/x86_64/os/repodata/repomd.xml: > > > > > > > > > > > > > > > > > > [Errno 14] curl#60 - "Peer's certificate issuer has been marked as > > > > > > not trusted by the user." > > > > > > : > > > > > > > > > > > > Is there a step I need to take on the clients so they'll accept this > > > > > > cert as trusted? I thought having it be signed by the IPA CA would > > > > > > have > > > > > > taken care of that. > > > > > > > > > > > > # ls -l /etc/ipa/ca.crt > > > > > > -rw-r--r-- 1 root root 2546 Apr 28 2014 /etc/ipa/ca.crt > > > > > > # > > > > > > > > > > Pretty much only IPA tools know to use this file. > > > > > > > > > > My knowledge is a bit stale on adding the IPA CA to the global trust > > > > > but I'm pretty sure it is done automatically now and I think it was in > > > > > the 4.2 timeframe. I'm assuming this is Fedora 21 so it doesn't have > > > > > this code. > > > > > > > > > > Look at this, > > > > > https://fedoraproject.org/wiki/Features/SharedSystemCertificates > > > > > > > >
Re: [Freeipa-users] Using our IPA CA as a trusted CA to sign ssl certificates
On 06/03/2016 11:02 AM, Rob Crittenden wrote: Bret Wortman wrote: I'm not sure I'd call what we have "success" just yet. ;-) You're right -- F21, IPA 4.1.4-1. I'll try the steps you outlined and see how we go. Rob, would you have just used the existing "localhost.key" instead of generating a new one? No, I think you did the right thing, the default keysize was probably still 1024 in F21. I double-checked the getcert-request man page and it looks like it will use an existing key if one exists in the key file passed in so I was wrong about that bit. You just didn't need to use req to generate a CSR as certmonger will do that for you. Good to know. I tried the update-ca-trust on both the yum server and on my workstation but nothing changed even after an httpd restart. I did take a peek inside /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt and didn't see my /etc/ipa/ca.crt in there (which may not be a problem, but I confess I'm not sure what should be where at this point). Bret rob On 06/03/2016 09:48 AM, Rob Crittenden wrote: Bret Wortman wrote: So for our internal yum server, I created a new key and cert request (it had a localhost key and cert but I wanted to start clean): # openssl genrsa 2048 > /etc/pki/tls/private/server.key # openssl req -new -x509 -nodes -sha1 -days 365 -key /etc/pki/tls/private/server.key > /etc/pki/tls/certs/server.crt # ipa-getcert request -f /etc/pki/tls/certs/server.crt -k /etc/pki/tls/private/server.key -r I try not to argue with success but I'd be curious what is actually going on here. You generate a CSR and call it a certificate. It is probably the case that certmonger is ignoring it altogether and generating its own CSR. ipa-getcert list shows it approved. I set up SSL in apache to use the above .key and .crt, but when I try to run yum against this using ssl: # yum search ffmpeg Loaded plugins: langpacks https://yum.private.net/fedora/releases/21/Everything/x86_64/os/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's certificate issuer has been marked as not trusted by the user." : Is there a step I need to take on the clients so they'll accept this cert as trusted? I thought having it be signed by the IPA CA would have taken care of that. # ls -l /etc/ipa/ca.crt -rw-r--r-- 1 root root 2546 Apr 28 2014 /etc/ipa/ca.crt # Pretty much only IPA tools know to use this file. My knowledge is a bit stale on adding the IPA CA to the global trust but I'm pretty sure it is done automatically now and I think it was in the 4.2 timeframe. I'm assuming this is Fedora 21 so it doesn't have this code. Look at this, https://fedoraproject.org/wiki/Features/SharedSystemCertificates The idea is to add the IPA CA to that and then all tools using SSL would "just work". Something like: # cp /etc/ipa/ca.crt /usr/share/pki/ca-trust-source/anchors/ipa-ca.pem # update-ca-trust You'd need to remember to manually undo this if you ever redo your IPA install (and get a new CA): # rm /etc/ipa/ca.crt /usr/share/pki/ca-trust-source/anchors/ipa-ca.pem # update-ca-trust Like I said, I'm pretty sure this is all automatic in some more recent versions of IPA. rob --- Bret On 06/02/2016 07:25 PM, bret.wort...@damascusgrp.com wrote: Cool. I'll give this a go in the morning. Bret Wortman http://wrapbuddies.co/ On Jun 2, 2016, 6:24 PM -0400, Fraser Tweedale <ftwee...@redhat.com>, wrote: On Thu, Jun 02, 2016 at 05:35:01PM -0400, bret.wort...@damascusgrp.com wrote: Sorry, let me back up a step. We need to implement hype everywhere. All our web services. And clients need to get keys automatically whether through IPA or Puppet. These systems use IPA for everything but authentication (to keep most users off). I'm trying to wuss out the easiest way to make this happen smoothly. Hi Bret, You can use the IPA CA to sign service certificates. See http://www.freeipa.org/page/Certmonger#Request_a_new_certificate. IPA-enrolled machines already have the IPA certificate in their trust store. If the clients are IPA-enrolled, everything should Just Work, otherwise you can distribute the IPA CA certificate to clients via Puppet** or whatever means you prefer. ** you will have to work out how, because I do not know Puppet :) Cheers, Fraser On Jun 2, 2016, 5:31 PM -0400, Rob Crittenden<rcrit...@redhat.com>, wrote: Bret Wortman wrote: Is it possible to use our freeipa CA as a trusted CA to sign our internal SSL certificates? Our system runs on a private network and so using the usual trusted sources isn't an option. We've been using self-signed, but that adds some additional complications and we thought this might be a good solution. Is it possible, and, since most online guides defer to "submit the CSR to Verisign" or whomever, how would you go about producing one in this way? Not sure I understand the question. The IPA CA is
Re: [Freeipa-users] Using our IPA CA as a trusted CA to sign ssl certificates
I'm not sure I'd call what we have "success" just yet. ;-) You're right -- F21, IPA 4.1.4-1. I'll try the steps you outlined and see how we go. Rob, would you have just used the existing "localhost.key" instead of generating a new one? On 06/03/2016 09:48 AM, Rob Crittenden wrote: Bret Wortman wrote: So for our internal yum server, I created a new key and cert request (it had a localhost key and cert but I wanted to start clean): # openssl genrsa 2048 > /etc/pki/tls/private/server.key # openssl req -new -x509 -nodes -sha1 -days 365 -key /etc/pki/tls/private/server.key > /etc/pki/tls/certs/server.crt # ipa-getcert request -f /etc/pki/tls/certs/server.crt -k /etc/pki/tls/private/server.key -r I try not to argue with success but I'd be curious what is actually going on here. You generate a CSR and call it a certificate. It is probably the case that certmonger is ignoring it altogether and generating its own CSR. ipa-getcert list shows it approved. I set up SSL in apache to use the above .key and .crt, but when I try to run yum against this using ssl: # yum search ffmpeg Loaded plugins: langpacks https://yum.private.net/fedora/releases/21/Everything/x86_64/os/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's certificate issuer has been marked as not trusted by the user." : Is there a step I need to take on the clients so they'll accept this cert as trusted? I thought having it be signed by the IPA CA would have taken care of that. # ls -l /etc/ipa/ca.crt -rw-r--r-- 1 root root 2546 Apr 28 2014 /etc/ipa/ca.crt # Pretty much only IPA tools know to use this file. My knowledge is a bit stale on adding the IPA CA to the global trust but I'm pretty sure it is done automatically now and I think it was in the 4.2 timeframe. I'm assuming this is Fedora 21 so it doesn't have this code. Look at this, https://fedoraproject.org/wiki/Features/SharedSystemCertificates The idea is to add the IPA CA to that and then all tools using SSL would "just work". Something like: # cp /etc/ipa/ca.crt /usr/share/pki/ca-trust-source/anchors/ipa-ca.pem # update-ca-trust You'd need to remember to manually undo this if you ever redo your IPA install (and get a new CA): # rm /etc/ipa/ca.crt /usr/share/pki/ca-trust-source/anchors/ipa-ca.pem # update-ca-trust Like I said, I'm pretty sure this is all automatic in some more recent versions of IPA. rob --- Bret On 06/02/2016 07:25 PM, bret.wort...@damascusgrp.com wrote: Cool. I'll give this a go in the morning. Bret Wortman http://wrapbuddies.co/ On Jun 2, 2016, 6:24 PM -0400, Fraser Tweedale <ftwee...@redhat.com>, wrote: On Thu, Jun 02, 2016 at 05:35:01PM -0400, bret.wort...@damascusgrp.com wrote: Sorry, let me back up a step. We need to implement hype everywhere. All our web services. And clients need to get keys automatically whether through IPA or Puppet. These systems use IPA for everything but authentication (to keep most users off). I'm trying to wuss out the easiest way to make this happen smoothly. Hi Bret, You can use the IPA CA to sign service certificates. See http://www.freeipa.org/page/Certmonger#Request_a_new_certificate. IPA-enrolled machines already have the IPA certificate in their trust store. If the clients are IPA-enrolled, everything should Just Work, otherwise you can distribute the IPA CA certificate to clients via Puppet** or whatever means you prefer. ** you will have to work out how, because I do not know Puppet :) Cheers, Fraser On Jun 2, 2016, 5:31 PM -0400, Rob Crittenden<rcrit...@redhat.com>, wrote: Bret Wortman wrote: Is it possible to use our freeipa CA as a trusted CA to sign our internal SSL certificates? Our system runs on a private network and so using the usual trusted sources isn't an option. We've been using self-signed, but that adds some additional complications and we thought this might be a good solution. Is it possible, and, since most online guides defer to "submit the CSR to Verisign" or whomever, how would you go about producing one in this way? Not sure I understand the question. The IPA CA is also self-signed. For enrolled systems though at least the CA is pre-distributed so maybe that will help. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Using our IPA CA as a trusted CA to sign ssl certificates
So for our internal yum server, I created a new key and cert request (it had a localhost key and cert but I wanted to start clean): # openssl genrsa 2048 > /etc/pki/tls/private/server.key # openssl req -new -x509 -nodes -sha1 -days 365 -key /etc/pki/tls/private/server.key > /etc/pki/tls/certs/server.crt # ipa-getcert request -f /etc/pki/tls/certs/server.crt -k /etc/pki/tls/private/server.key -r ipa-getcert list shows it approved. I set up SSL in apache to use the above .key and .crt, but when I try to run yum against this using ssl: # yum search ffmpeg Loaded plugins: langpacks https://yum.private.net/fedora/releases/21/Everything/x86_64/os/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's certificate issuer has been marked as not trusted by the user." : Is there a step I need to take on the clients so they'll accept this cert as trusted? I thought having it be signed by the IPA CA would have taken care of that. # ls -l /etc/ipa/ca.crt -rw-r--r-- 1 root root 2546 Apr 28 2014 /etc/ipa/ca.crt # --- Bret On 06/02/2016 07:25 PM, bret.wort...@damascusgrp.com wrote: Cool. I'll give this a go in the morning. Bret Wortman http://wrapbuddies.co/ On Jun 2, 2016, 6:24 PM -0400, Fraser Tweedale <ftwee...@redhat.com>, wrote: On Thu, Jun 02, 2016 at 05:35:01PM -0400, bret.wort...@damascusgrp.com wrote: Sorry, let me back up a step. We need to implement hype everywhere. All our web services. And clients need to get keys automatically whether through IPA or Puppet. These systems use IPA for everything but authentication (to keep most users off). I'm trying to wuss out the easiest way to make this happen smoothly. Hi Bret, You can use the IPA CA to sign service certificates. See http://www.freeipa.org/page/Certmonger#Request_a_new_certificate. IPA-enrolled machines already have the IPA certificate in their trust store. If the clients are IPA-enrolled, everything should Just Work, otherwise you can distribute the IPA CA certificate to clients via Puppet** or whatever means you prefer. ** you will have to work out how, because I do not know Puppet :) Cheers, Fraser On Jun 2, 2016, 5:31 PM -0400, Rob Crittenden<rcrit...@redhat.com>, wrote: Bret Wortman wrote: Is it possible to use our freeipa CA as a trusted CA to sign our internal SSL certificates? Our system runs on a private network and so using the usual trusted sources isn't an option. We've been using self-signed, but that adds some additional complications and we thought this might be a good solution. Is it possible, and, since most online guides defer to "submit the CSR to Verisign" or whomever, how would you go about producing one in this way? Not sure I understand the question. The IPA CA is also self-signed. For enrolled systems though at least the CA is pre-distributed so maybe that will help. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Using our IPA CA as a trusted CA to sign ssl certificates
Cool. I'll give this a go in the morning. Bret Wortman http://wrapbuddies.co/ On Jun 2, 2016, 6:24 PM -0400, Fraser Tweedale<ftwee...@redhat.com>, wrote: > On Thu, Jun 02, 2016 at 05:35:01PM -0400, bret.wort...@damascusgrp.com wrote: > > Sorry, let me back up a step. We need to implement hype > > everywhere. All our web services. And clients need to get > > keys automatically whether through IPA or Puppet. These > > systems use IPA for everything but authentication (to keep most > > users off). I'm trying to wuss out the easiest way to make this > > happen smoothly. > > > Hi Bret, > > You can use the IPA CA to sign service certificates. See > http://www.freeipa.org/page/Certmonger#Request_a_new_certificate. > > IPA-enrolled machines already have the IPA certificate in their > trust store. If the clients are IPA-enrolled, everything should > Just Work, otherwise you can distribute the IPA CA certificate to > clients via Puppet** or whatever means you prefer. > > ** you will have to work out how, because I do not know Puppet :) > > Cheers, > Fraser > > > > > > > On Jun 2, 2016, 5:31 PM -0400, Rob Crittenden<rcrit...@redhat.com>, wrote: > > > Bret Wortman wrote: > > > > Is it possible to use our freeipa CA as a trusted CA to sign our > > > > internal SSL certificates? Our system runs on a private network and so > > > > using the usual trusted sources isn't an option. We've been using > > > > self-signed, but that adds some additional complications and we thought > > > > this might be a good solution. > > > > > > > > Is it possible, and, since most online guides defer to "submit the CSR > > > > to Verisign" or whomever, how would you go about producing one in this > > > > way? > > > > > > Not sure I understand the question. The IPA CA is also self-signed. For > > > enrolled systems though at least the CA is pre-distributed so maybe that > > > will help. > > > > > > rob > > > > > > -- > > Manage your subscription for the Freeipa-users mailing list: > > https://www.redhat.com/mailman/listinfo/freeipa-users > > Go to http://freeipa.org for more info on the project > -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Scratch that. Decided to be daring and run "getcert resubmit -i" for each cert (after verifying the first one worked), then shut ipa down, advanced the date, re-enabled ntpd and started it back up. Looks clean. On 04/29/2016 01:22 PM, Bret Wortman wrote: Of course, I just remembered that the server still thinks it's April 4, and I still have some certs that are expiring as of 4-17-16. Before I screw anything else up, what's the RIGHT way to renew those certs and move the server back to real time? On 04/29/2016 01:07 PM, Bret Wortman wrote: Hot damn! It's up and running. Web UI works. CLI works. The chgrp did the trick. Thank you Rob, Petr and Christian! Bret On 04/29/2016 01:04 PM, Rob Crittenden wrote: Bret Wortman wrote: We run with selinux disabled. # getenforce Disabled # restorecon -R -v /etc/httpd/alias # ipactl start Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Starting ipa_memcached Service Starting httpd Service Starting pki-tomcatd Service Failed to start pki-tomcatd Service Shutting down Aborting ipactl # ipactl status Directory Service: STOPPED Directory Service must be running in order to obtain status of other services ipa: INFO: The ipactl command was successful # The problem is permissions. Try: # chgrp apache /etc/httpd/alias/*.db The mode is ok, Apache only needs read access. The segfault is fixed upstream and actual usable error messages reported. The init system doesn't see it as a failure because this happens after Apache forks its children. I'd also consider re-enabling SELinux eventually. rob On 04/29/2016 12:25 PM, Christian Heimes wrote: On 2016-04-29 18:17, Bret Wortman wrote: I'll put the results inline here, since they're short. [root@zsipa log]# ls -laZ /etc/httpd/ drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 . drwxr-xr-x. root root system_u:object_r:etc_t:s0 .. drwxr-xr-x. root root system_u:object_r:cert_t:s0 alias drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.d drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.modules.d lrwxrwxrwx root root ? logs -> ../../var/log/httpd lrwxrwxrwx root root ? modules -> ../../usr/lib64/httpd/modules lrwxrwxrwx root root ? run -> /run/httpd [root@zsipa log]# ls -laZ /etc/httpd/alias drwxr-xr-x. root root system_u:object_r:cert_t:s0 . drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 .. -r--r--r-- root root ? cacert.asc -r--r--r-- root root ? cacert.asc.orig -rw-r- root root ? cert8.db -rw-rw root apache ? cert8.db.20160426 -rw-rw root apache ? cert8.db.orig -rw---. root root system_u:object_r:cert_t:s0 install.log -rw-r- root root ? key3.db -rw-rw root apache ? key3.db.20160426 -rw-rw root apache ? key3.db.orig lrwxrwxrwx root root ? libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw root apache ? pwdfile.txt -rw-rw root apache ? pwdfile.txt.orig -rw-rw root apache ? secmod.db -rw-rw root apache ? secmod.db.orig Some files don't have the correct SELinux context or are completely missing a context. SELinux prevents Apache from accessing this files. Did you replace some files or restore some from a backup? You should see a bunch of SELinux violations in your audit log. In order to restore the correct context, please run restorecon: # restorecon -R -v /etc/httpd/alias This should set correct contexts and allow you to start Apache HTTPD again. Christian -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Of course, I just remembered that the server still thinks it's April 4, and I still have some certs that are expiring as of 4-17-16. Before I screw anything else up, what's the RIGHT way to renew those certs and move the server back to real time? On 04/29/2016 01:07 PM, Bret Wortman wrote: Hot damn! It's up and running. Web UI works. CLI works. The chgrp did the trick. Thank you Rob, Petr and Christian! Bret On 04/29/2016 01:04 PM, Rob Crittenden wrote: Bret Wortman wrote: We run with selinux disabled. # getenforce Disabled # restorecon -R -v /etc/httpd/alias # ipactl start Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Starting ipa_memcached Service Starting httpd Service Starting pki-tomcatd Service Failed to start pki-tomcatd Service Shutting down Aborting ipactl # ipactl status Directory Service: STOPPED Directory Service must be running in order to obtain status of other services ipa: INFO: The ipactl command was successful # The problem is permissions. Try: # chgrp apache /etc/httpd/alias/*.db The mode is ok, Apache only needs read access. The segfault is fixed upstream and actual usable error messages reported. The init system doesn't see it as a failure because this happens after Apache forks its children. I'd also consider re-enabling SELinux eventually. rob On 04/29/2016 12:25 PM, Christian Heimes wrote: On 2016-04-29 18:17, Bret Wortman wrote: I'll put the results inline here, since they're short. [root@zsipa log]# ls -laZ /etc/httpd/ drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 . drwxr-xr-x. root root system_u:object_r:etc_t:s0 .. drwxr-xr-x. root root system_u:object_r:cert_t:s0 alias drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.d drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.modules.d lrwxrwxrwx root root ? logs -> ../../var/log/httpd lrwxrwxrwx root root ? modules -> ../../usr/lib64/httpd/modules lrwxrwxrwx root root ?run -> /run/httpd [root@zsipa log]# ls -laZ /etc/httpd/alias drwxr-xr-x. root root system_u:object_r:cert_t:s0 . drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 .. -r--r--r-- root root ? cacert.asc -r--r--r-- root root ? cacert.asc.orig -rw-r- root root ? cert8.db -rw-rw root apache ? cert8.db.20160426 -rw-rw root apache ? cert8.db.orig -rw---. root root system_u:object_r:cert_t:s0 install.log -rw-r- root root ? key3.db -rw-rw root apache ? key3.db.20160426 -rw-rw root apache ? key3.db.orig lrwxrwxrwx root root ? libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw root apache ? pwdfile.txt -rw-rw root apache ? pwdfile.txt.orig -rw-rw root apache ? secmod.db -rw-rw root apache ? secmod.db.orig Some files don't have the correct SELinux context or are completely missing a context. SELinux prevents Apache from accessing this files. Did you replace some files or restore some from a backup? You should see a bunch of SELinux violations in your audit log. In order to restore the correct context, please run restorecon: # restorecon -R -v /etc/httpd/alias This should set correct contexts and allow you to start Apache HTTPD again. Christian -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Hot damn! It's up and running. Web UI works. CLI works. The chgrp did the trick. Thank you Rob, Petr and Christian! Bret On 04/29/2016 01:04 PM, Rob Crittenden wrote: Bret Wortman wrote: We run with selinux disabled. # getenforce Disabled # restorecon -R -v /etc/httpd/alias # ipactl start Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Starting ipa_memcached Service Starting httpd Service Starting pki-tomcatd Service Failed to start pki-tomcatd Service Shutting down Aborting ipactl # ipactl status Directory Service: STOPPED Directory Service must be running in order to obtain status of other services ipa: INFO: The ipactl command was successful # The problem is permissions. Try: # chgrp apache /etc/httpd/alias/*.db The mode is ok, Apache only needs read access. The segfault is fixed upstream and actual usable error messages reported. The init system doesn't see it as a failure because this happens after Apache forks its children. I'd also consider re-enabling SELinux eventually. rob On 04/29/2016 12:25 PM, Christian Heimes wrote: On 2016-04-29 18:17, Bret Wortman wrote: I'll put the results inline here, since they're short. [root@zsipa log]# ls -laZ /etc/httpd/ drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 . drwxr-xr-x. root root system_u:object_r:etc_t:s0 .. drwxr-xr-x. root root system_u:object_r:cert_t:s0 alias drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.d drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.modules.d lrwxrwxrwx root root ?logs -> ../../var/log/httpd lrwxrwxrwx root root ? modules -> ../../usr/lib64/httpd/modules lrwxrwxrwx root root ?run -> /run/httpd [root@zsipa log]# ls -laZ /etc/httpd/alias drwxr-xr-x. root root system_u:object_r:cert_t:s0 . drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 .. -r--r--r-- root root ? cacert.asc -r--r--r-- root root ? cacert.asc.orig -rw-r- root root ? cert8.db -rw-rw root apache ? cert8.db.20160426 -rw-rw root apache ? cert8.db.orig -rw---. root root system_u:object_r:cert_t:s0 install.log -rw-r- root root ? key3.db -rw-rw root apache ? key3.db.20160426 -rw-rw root apache ? key3.db.orig lrwxrwxrwx root root ? libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw root apache ? pwdfile.txt -rw-rw root apache ? pwdfile.txt.orig -rw-rw root apache ? secmod.db -rw-rw root apache ? secmod.db.orig Some files don't have the correct SELinux context or are completely missing a context. SELinux prevents Apache from accessing this files. Did you replace some files or restore some from a backup? You should see a bunch of SELinux violations in your audit log. In order to restore the correct context, please run restorecon: # restorecon -R -v /etc/httpd/alias This should set correct contexts and allow you to start Apache HTTPD again. Christian -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
We run with selinux disabled. # getenforce Disabled # restorecon -R -v /etc/httpd/alias # ipactl start Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Starting ipa_memcached Service Starting httpd Service Starting pki-tomcatd Service Failed to start pki-tomcatd Service Shutting down Aborting ipactl # ipactl status Directory Service: STOPPED Directory Service must be running in order to obtain status of other services ipa: INFO: The ipactl command was successful # On 04/29/2016 12:25 PM, Christian Heimes wrote: On 2016-04-29 18:17, Bret Wortman wrote: I'll put the results inline here, since they're short. [root@zsipa log]# ls -laZ /etc/httpd/ drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 . drwxr-xr-x. root root system_u:object_r:etc_t:s0 .. drwxr-xr-x. root root system_u:object_r:cert_t:s0 alias drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.d drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.modules.d lrwxrwxrwx root root ?logs -> ../../var/log/httpd lrwxrwxrwx root root ?modules -> ../../usr/lib64/httpd/modules lrwxrwxrwx root root ?run -> /run/httpd [root@zsipa log]# ls -laZ /etc/httpd/alias drwxr-xr-x. root root system_u:object_r:cert_t:s0 . drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 .. -r--r--r-- root root ?cacert.asc -r--r--r-- root root ?cacert.asc.orig -rw-r- root root ?cert8.db -rw-rw root apache ?cert8.db.20160426 -rw-rw root apache ?cert8.db.orig -rw---. root root system_u:object_r:cert_t:s0 install.log -rw-r- root root ?key3.db -rw-rw root apache ?key3.db.20160426 -rw-rw root apache ?key3.db.orig lrwxrwxrwx root root ?libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw root apache ?pwdfile.txt -rw-rw root apache ?pwdfile.txt.orig -rw-rw root apache ?secmod.db -rw-rw root apache ?secmod.db.orig Some files don't have the correct SELinux context or are completely missing a context. SELinux prevents Apache from accessing this files. Did you replace some files or restore some from a backup? You should see a bunch of SELinux violations in your audit log. In order to restore the correct context, please run restorecon: # restorecon -R -v /etc/httpd/alias This should set correct contexts and allow you to start Apache HTTPD again. Christian -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
I'll put the results inline here, since they're short.
[root@zsipa log]# ls -laZ /etc/httpd/
drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 .
drwxr-xr-x. root root system_u:object_r:etc_t:s0 ..
drwxr-xr-x. root root system_u:object_r:cert_t:s0 alias
drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf
drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.d
drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 conf.modules.d
lrwxrwxrwx root root ?logs ->
../../var/log/httpd
lrwxrwxrwx root root ?modules ->
../../usr/lib64/httpd/modules
lrwxrwxrwx root root ?run -> /run/httpd
[root@zsipa log]# ls -laZ /etc/httpd/alias
drwxr-xr-x. root root system_u:object_r:cert_t:s0 .
drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 ..
-r--r--r-- root root ? cacert.asc
-r--r--r-- root root ? cacert.asc.orig
-rw-r- root root ?cert8.db
-rw-rw root apache ? cert8.db.20160426
-rw-rw root apache ? cert8.db.orig
-rw---. root root system_u:object_r:cert_t:s0 install.log
-rw-r- root root ?key3.db
-rw-rw root apache ? key3.db.20160426
-rw-rw root apache ? key3.db.orig
lrwxrwxrwx root root ? libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so
-rw-rw root apache ? pwdfile.txt
-rw-rw root apache ? pwdfile.txt.orig
-rw-rw root apache ?secmod.db
-rw-rw root apache ? secmod.db.orig
[root@zsipa log]# certutil -L -d /etc/httpd/alias
Certificate Nickname Trust
Attributes
SSL,S/MIME,JAR/XPI
Signing-Cert u,u,u
Server-Cert u,u,u
ipaCert u,u,u
PRIVATE.NET IPA CA CT,C,C
PRIVATE.NET IPA CA CT,C,C
[root@zsipa log]#
On 04/29/2016 11:02 AM, Christian Heimes wrote:
On 2016-04-29 16:51, Bret Wortman wrote:
It is contacting the correct machine. I tried again by IP with the same
results.
/etc/httpd/conf.d/ipa-pki-proxy.conf is dated May 20 2014.
Web UI won't load. CLI won't respond either. Commands just hang.
# netstat -ln | grep 443
tcp6 0 0 :::8443
:::* LISTEN
tcp6 2 0 :::443
:::* LISTEN
# netstat -ln | grep 8009
tcp6 0 0 127.0.0.1:8009
:::* LISTEN
# curl -v https://zsipa.private.net:443/ca/admin/ca/getStatus
* Hostname was NOT found in DNS cache
* Trying 192.168.208.53...
* Connected to zsipa.private.net (192.168.208.53) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
(long hang at this point, so I ^C-ed)
# openssl s_client -connect zsipa.private.net:443 -CAfile
/etc/ipa/ca.crt -verify 10
verify depth is 10
CONNECTED(0003)
(long hang at this point, aborted again)
For the other (longer) logs, see http://pastebin.com/esBBKyGZ
Also, answering Christian's questions:
mod_ssl has not been installed.
# ss -tpln | grep 443
LISTEN 0 100:::8443 :::*
users:(("java",pid=26522,fd=84))
LISTEN 13 128:::443:::*
users:(("httpd",pid=26323,fd=6))
#
The output of ss looks sane. httpd is Apache, Java is Dogtag PKI's
Tomcat instance.
The error log of Apache is more troublesome. It looks like your NSSDB is
busted:
[Mon Apr 04 14:18:49.330238 2016] [:error] [pid 26327] NSS_Initialize
failed. Certificate database: /etc/httpd/alias.
[Mon Apr 04 14:18:49.330253 2016] [:error] [pid 26327] SSL Library
Error: -8038 SEC_ERROR_NOT_INITIALIZED
[Mon Apr 04 14:18:50.318327 2016] [core:notice] [pid 26323] AH00052:
child pid 26327 exit signal Segmentation fault (11)
Please run this commands to show us the content of your NSSDB.
# ls -laZ /etc/httpd/
# ls -laZ /etc/httpd/alias
# certutil -L -d /etc/httpd/alias
Christian
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
It is contacting the correct machine. I tried again by IP with the same
results.
/etc/httpd/conf.d/ipa-pki-proxy.conf is dated May 20 2014.
Web UI won't load. CLI won't respond either. Commands just hang.
# netstat -ln | grep 443
tcp6 0 0 :::8443 :::* LISTEN
tcp6 2 0 :::443:::* LISTEN
# netstat -ln | grep 8009
tcp6 0 0 127.0.0.1:8009 :::* LISTEN
# curl -v https://zsipa.private.net:443/ca/admin/ca/getStatus
* Hostname was NOT found in DNS cache
* Trying 192.168.208.53...
* Connected to zsipa.private.net (192.168.208.53) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
(long hang at this point, so I ^C-ed)
# openssl s_client -connect zsipa.private.net:443 -CAfile
/etc/ipa/ca.crt -verify 10
verify depth is 10
CONNECTED(0003)
(long hang at this point, aborted again)
For the other (longer) logs, see http://pastebin.com/esBBKyGZ
Also, answering Christian's questions:
mod_ssl has not been installed.
# ss -tpln | grep 443
LISTEN 0 100:::8443 :::*
users:(("java",pid=26522,fd=84))
LISTEN 13 128:::443:::*
users:(("httpd",pid=26323,fd=6))
#
On 04/29/2016 10:08 AM, Petr Vobornik wrote:
On 04/29/2016 02:53 PM, Bret Wortman wrote:
Despite "ipactl status" indicating that all processes were running after
step 1, step 2 produces "Unable to establish SSL connection."
Full terminal session is at http://pastebin.com/ZuNBHPy0
Hm, it doesn't help me much.
Does it contact the correct machine? I.e., is IP address OK?
What is the result of:
netstat -ln | grep 443
netstat -ln | grep 8009
Have you modified by any chance: /etc/httpd/conf.d/ipa-pki-proxy.conf
Try to run curl, maybe it will be more verbose, but probably not:
# curl -v https://zsipa.private.net:443/ca/admin/ca/getStatus
Christian(CCd), do you have any ideas?
Could you look into /var/log/httpd/error_log or syslog(would try
/var/log/message and journalctl), There might be more information about the:
"""
status: NEED_TO_SUBMIT
ca-error: Internal error
"""
Which may help us with root culprit.
Do web ui or CLI work?
On 04/29/2016 07:29 AM, Petr Vobornik wrote:
On 04/29/2016 12:03 PM, Bret Wortman wrote:
The date change was due (I think) to me changing the date back to 4/1
yesterday, though I left it there and haven't updated it again until
this morning, when I went back to 4/1 again.
I put the results of the commands you requested at
https://pastebin.com/s7cHAh6R. Thanks for your help, Petr. I really
appreciate it.
Bret
If I combine this and the previous output, it seems that:
- PKI starts normally
- ipactl has troubles with determining that PKI started and after 5mins
of failed attempts it stops whole IPA (expected behavior when a service
doesn't start)
The failed attempt is:
"""
ipa: DEBUG: Waiting until the CA is running
ipa: DEBUG: Starting external process
ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30'
'--no-check-certificate'
'https://zsipa.private.net:443/ca/admin/ca/getStatus'
ipa: DEBUG: Process finished, return code=4
ipa: DEBUG: stdout=
ipa: DEBUG: stderr=--2016-04-01 09:39:50--
https://zsipa.private.net/ca/admin/ca/getStatus
Resolving zsipa.private.net (zsipa.private.net)... 192.168.208.53
Connecting to zsipa.private.net
(zsipa.private.net)|192.168.208.53|:443... connected.
Unable to establish SSL connection.
ipa: DEBUG: The CA status is: check interrupted due to error: Command
''/usr/bin/wget' '-S' '-O' '-' '--timeout=30' '--no-check-certificate'
'https://zsipa.private.net:443/ca/admin/ca/getStatus'' returned non-zero
exit status 4
"""
It says "Unable to establish SSL connection", it would be good to get
more details.
Also given that the CA cert was renewed on April 3rd and that all certs
expires after that date, we should rather use date April 4th when moving
the date back.
So first start IPA again (date April 4th) but force it to not stop
services
1. ipactl start --force
wait until all is started
2. wget -v -d -S -O - --timeout=30 --no-check-certificate
https://zsipa.private.net:443/ca/admin/ca/getStatus
optionally (assuming that CA won't be turned of)
3. getcert list
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Despite "ipactl status" indicating that all processes were running after step 1, step 2 produces "Unable to establish SSL connection." Full terminal session is at http://pastebin.com/ZuNBHPy0 On 04/29/2016 07:29 AM, Petr Vobornik wrote: On 04/29/2016 12:03 PM, Bret Wortman wrote: The date change was due (I think) to me changing the date back to 4/1 yesterday, though I left it there and haven't updated it again until this morning, when I went back to 4/1 again. I put the results of the commands you requested at https://pastebin.com/s7cHAh6R. Thanks for your help, Petr. I really appreciate it. Bret If I combine this and the previous output, it seems that: - PKI starts normally - ipactl has troubles with determining that PKI started and after 5mins of failed attempts it stops whole IPA (expected behavior when a service doesn't start) The failed attempt is: """ ipa: DEBUG: Waiting until the CA is running ipa: DEBUG: Starting external process ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30' '--no-check-certificate' 'https://zsipa.private.net:443/ca/admin/ca/getStatus' ipa: DEBUG: Process finished, return code=4 ipa: DEBUG: stdout= ipa: DEBUG: stderr=--2016-04-01 09:39:50-- https://zsipa.private.net/ca/admin/ca/getStatus Resolving zsipa.private.net (zsipa.private.net)... 192.168.208.53 Connecting to zsipa.private.net (zsipa.private.net)|192.168.208.53|:443... connected. Unable to establish SSL connection. ipa: DEBUG: The CA status is: check interrupted due to error: Command ''/usr/bin/wget' '-S' '-O' '-' '--timeout=30' '--no-check-certificate' 'https://zsipa.private.net:443/ca/admin/ca/getStatus'' returned non-zero exit status 4 """ It says "Unable to establish SSL connection", it would be good to get more details. Also given that the CA cert was renewed on April 3rd and that all certs expires after that date, we should rather use date April 4th when moving the date back. So first start IPA again (date April 4th) but force it to not stop services 1. ipactl start --force wait until all is started 2. wget -v -d -S -O - --timeout=30 --no-check-certificate https://zsipa.private.net:443/ca/admin/ca/getStatus optionally (assuming that CA won't be turned of) 3. getcert list -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
The date change was due (I think) to me changing the date back to 4/1
yesterday, though I left it there and haven't updated it again until
this morning, when I went back to 4/1 again.
I put the results of the commands you requested at
https://pastebin.com/s7cHAh6R. Thanks for your help, Petr. I really
appreciate it.
Bret
On 04/29/2016 04:59 AM, Petr Vobornik wrote:
comments inline
On 04/28/2016 06:30 PM, Bret Wortman wrote:
Look, I'll be honest. When IPA is in this much of a knot, I don't know how to do
the simplest things with its various components. For example, I've no clue how
to search the ldap database for anything. Or even how to authenticate since
Kerberos isn't running. IPA has sheltered me from ldap for so long that it's a
problem at times like this.
That being said, here are the things I /was/ able to handle:
Apr 01 11:02:40 zsipa.private.net server[6896]: Java virtual machine used:
/usr/lib/jvm/jre/bin/java
Apr 01 11:02:40 zsipa.private.net server[6896]: classpath used:
/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/lib/java/commons-daemon.j
Apr 01 11:02:40 zsipa.private.net server[6896]: main class used:
org.apache.catalina.startup.Bootstrap
Apr 01 11:02:40 zsipa.private.net server[6896]: flags used:
-DRESTEASY_LIB=/usr/share/java/resteasy
Apr 01 11:02:40 zsipa.private.net server[6896]: options used:
-Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat
-Djava.endorsed.dirs= -Djava.io.
Apr 01 11:02:40 zsipa.private.net server[6896]: arguments used: start
Apr 01 11:02:40 zsipa.private.net server[6896]: Apr 01, 2016 11:02:40 AM
org.apache.catalina.startup.ClassLoaderFactory validateFile
Apr 01 11:02:40 zsipa.private.net server[6896]: WARNING: Problem with JAR file
[/var/lib/pki/pki-tomcat/lib/log4j.jar], exists: [false], canRead: [false]
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property 'enableOCSP'
to 'false' did not find a matchi
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property
'ocspResponderURL' to 'http://zsipa.private.net:9
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property
'ocspResponderCertNickname' to 'ocspSigningCe
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property
'ocspCacheSize' to '1000' did not find a matc
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property
'ocspMinCacheEntryDuration' to '60' did not f
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property
'ocspMaxCacheEntryDuration' to '120' did not
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspTimeout'
to '10' did not find a matching
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property
'strictCiphers' to 'true' did not find a matc
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslOptions'
to 'ssl2=true,ssl3=true,tls=true
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896]: WARNING:
[SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ssl2Ciphers'
to '-SSL2_RC4_128_WITH_MD5,-SSL
Apr 01 11:02:41 zsipa.private.net server[6896]: Apr 01, 2016 11:02:41 AM
org.apache.catalina.startup.SetAllPropertiesRule begin
Apr 01 11:02:41 zsipa.private.net server[6896
Re: [Freeipa-users] IPA server having cert issues
Okay, I ran 'ldapsearch -x -h zsipa -p 389 -b 'ou=people,o=ipaca' and dumped that to a file. I'm still not clear on what I'm supposed to be looking for in the output, though. The result of systemctl | grep dirsrv@ was pretty uninformative. If the answer was "dirsrv", then I don't find that in the ldapsearch results. Assuming that was the ldapsearch command I needed to run On 04/28/2016 12:04 PM, Petr Vobornik wrote: On 04/28/2016 05:49 PM, Bret Wortman wrote: My system shows pki-server is installed and V10.2.1-3.fc21, but I don't have the pki-server binary itself. Will reinstalling this rpm hurt me in any way? Without it, I'm not sure how to check my system against the messages you provided below. Not sure what you mean. Running doesn't require any additional packages. It is just to get additional logs. systemctl status pki-tomcatd@pki-tomcat.service journalctl -u pki-tomcatd@pki-tomcat.service And the links below are about checking if CA users have correctly mapped certificates in LDAP database in ou=people,o=ipaca for that you need only ldapsearch command and start directory server: systemctl start dirsrv@YOUR-REALM-TEST.service Proper name for dirsrv@YOUR-REALM-TEST.service can be found using: systemctl | grep dirsrv@ On 04/28/2016 11:07 AM, Petr Vobornik wrote: On 04/28/2016 04:07 PM, Bret Wortman wrote: Okay. This morning, I turned back time to 4/1 and started up IPA. It didn't work, but I got something new and interesting in the debug log, which I've posted to http://pastebin.com/M9VGCS8A. Lots of garbled junk came pouring out which doesn't happen when I'm set to real time. Is /this/ significant? Anything in systemctl status pki-tomcatd@pki-tomcat.service or rather: journalctl -u pki-tomcatd@pki-tomcat.service ? Just to be sure, it might be also worth to check if CA subsystem users have correct certs assigned: * https://www.redhat.com/archives/freeipa-users/2016-April/msg00138.html * https://www.redhat.com/archives/freeipa-users/2016-April/msg00143.html On 04/27/2016 02:24 PM, Bret Wortman wrote: I put excerpts from the ca logs in http://pastebin.com/gYgskU79. It looks logical to me, but I can't spot anything that looks like a root cause error. The selftests are all okay, I think. The debug log might have something, but it might also just be complaining about ldap not being up because it's not. On 04/27/2016 01:11 PM, Rob Crittenden wrote: Bret Wortman wrote: So in lieu of fixing these certs, is there an acceptable way to dump them all and start over /without losing the contents of the IPA database/? Or otherwise really screwing ourselves? I don't believe there is a way. We have a replica that's still up and running and we've switched everyone over to talking to it, but we're at risk with just the one. I'd ignore the two unknown certs for now. They look like someone was experimenting with issuing a cert and didn't quite get things working. The CA seems to be throwing an error. I'd check the syslog for messages from certmonger and look at the CA debug log and selftest log. rob [snip] -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
ded (/usr/lib/systemd/system/pki-tomcatd@.service; enabled) Active: inactive (dead) Apr 28 12:12:53 zsipa.private.net server[8557]: Apr 28, 2016 12:12:53 PM org.apache.catalina.core.StandardServer await Apr 28 12:12:53 zsipa.private.net server[8557]: INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance. Apr 28 12:12:53 zsipa.private.net server[8557]: Apr 28, 2016 12:12:53 PM org.apache.coyote.AbstractProtocol pause Apr 28 12:12:53 zsipa.private.net server[8557]: INFO: Pausing ProtocolHandler ["http-bio-8080"] Apr 28 12:12:53 zsipa.private.net server[8557]: Apr 28, 2016 12:12:53 PM org.apache.coyote.AbstractProtocol pause Apr 28 12:12:53 zsipa.private.net server[8557]: INFO: Pausing ProtocolHandler ["http-bio-8443"] Apr 28 12:12:53 zsipa.private.net server[8557]: Apr 28, 2016 12:12:53 PM org.apache.coyote.AbstractProtocol pause Apr 28 12:12:53 zsipa.private.net server[8557]: INFO: Pausing ProtocolHandler ["ajp-bio-127.0.0.1-8009"] Apr 28 12:12:53 zsipa.private.net server[8557]: Apr 28, 2016 12:12:53 PM org.apache.catalina.core.StandardService stopInternal Apr 28 12:12:53 zsipa.private.net server[8557]: INFO: Stopping service Catalina # systemctl | grep dirsrv@ dirsrv@PRIVATE-NET.service loaded active running 389 Directory Server PRIVATE-NET. On 04/28/2016 12:04 PM, Petr Vobornik wrote: On 04/28/2016 05:49 PM, Bret Wortman wrote: My system shows pki-server is installed and V10.2.1-3.fc21, but I don't have the pki-server binary itself. Will reinstalling this rpm hurt me in any way? Without it, I'm not sure how to check my system against the messages you provided below. Not sure what you mean. Running doesn't require any additional packages. It is just to get additional logs. systemctl status pki-tomcatd@pki-tomcat.service journalctl -u pki-tomcatd@pki-tomcat.service And the links below are about checking if CA users have correctly mapped certificates in LDAP database in ou=people,o=ipaca for that you need only ldapsearch command and start directory server: systemctl start dirsrv@YOUR-REALM-TEST.service Proper name for dirsrv@YOUR-REALM-TEST.service can be found using: systemctl | grep dirsrv@ On 04/28/2016 11:07 AM, Petr Vobornik wrote: On 04/28/2016 04:07 PM, Bret Wortman wrote: Okay. This morning, I turned back time to 4/1 and started up IPA. It didn't work, but I got something new and interesting in the debug log, which I've posted to http://pastebin.com/M9VGCS8A. Lots of garbled junk came pouring out which doesn't happen when I'm set to real time. Is /this/ significant? Anything in systemctl status pki-tomcatd@pki-tomcat.service or rather: journalctl -u pki-tomcatd@pki-tomcat.service ? Just to be sure, it might be also worth to check if CA subsystem users have correct certs assigned: * https://www.redhat.com/archives/freeipa-users/2016-April/msg00138.html * https://www.redhat.com/archives/freeipa-users/2016-April/msg00143.html On 04/27/2016 02:24 PM, Bret Wortman wrote: I put excerpts from the ca logs in http://pastebin.com/gYgskU79. It looks logical to me, but I can't spot anything that looks like a root cause error. The selftests are all okay, I think. The debug log might have something, but it might also just be complaining about ldap not being up because it's not. On 04/27/2016 01:11 PM, Rob Crittenden wrote: Bret Wortman wrote: So in lieu of fixing these certs, is there an acceptable way to dump them all and start over /without losing the contents of the IPA database/? Or otherwise really screwing ourselves? I don't believe there is a way. We have a replica that's still up and running and we've switched everyone over to talking to it, but we're at risk with just the one. I'd ignore the two unknown certs for now. They look like someone was experimenting with issuing a cert and didn't quite get things working. The CA seems to be throwing an error. I'd check the syslog for messages from certmonger and look at the CA debug log and selftest log. rob [snip] -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Okay. I got hung up on the first link doing some checking using pki-server. I don't see any reference to ldapsearch in either message, but I'll do what I can. On 04/28/2016 12:04 PM, Petr Vobornik wrote: On 04/28/2016 05:49 PM, Bret Wortman wrote: My system shows pki-server is installed and V10.2.1-3.fc21, but I don't have the pki-server binary itself. Will reinstalling this rpm hurt me in any way? Without it, I'm not sure how to check my system against the messages you provided below. Not sure what you mean. Running doesn't require any additional packages. It is just to get additional logs. systemctl status pki-tomcatd@pki-tomcat.service journalctl -u pki-tomcatd@pki-tomcat.service And the links below are about checking if CA users have correctly mapped certificates in LDAP database in ou=people,o=ipaca for that you need only ldapsearch command and start directory server: systemctl start dirsrv@YOUR-REALM-TEST.service Proper name for dirsrv@YOUR-REALM-TEST.service can be found using: systemctl | grep dirsrv@ On 04/28/2016 11:07 AM, Petr Vobornik wrote: On 04/28/2016 04:07 PM, Bret Wortman wrote: Okay. This morning, I turned back time to 4/1 and started up IPA. It didn't work, but I got something new and interesting in the debug log, which I've posted to http://pastebin.com/M9VGCS8A. Lots of garbled junk came pouring out which doesn't happen when I'm set to real time. Is /this/ significant? Anything in systemctl status pki-tomcatd@pki-tomcat.service or rather: journalctl -u pki-tomcatd@pki-tomcat.service ? Just to be sure, it might be also worth to check if CA subsystem users have correct certs assigned: * https://www.redhat.com/archives/freeipa-users/2016-April/msg00138.html * https://www.redhat.com/archives/freeipa-users/2016-April/msg00143.html On 04/27/2016 02:24 PM, Bret Wortman wrote: I put excerpts from the ca logs in http://pastebin.com/gYgskU79. It looks logical to me, but I can't spot anything that looks like a root cause error. The selftests are all okay, I think. The debug log might have something, but it might also just be complaining about ldap not being up because it's not. On 04/27/2016 01:11 PM, Rob Crittenden wrote: Bret Wortman wrote: So in lieu of fixing these certs, is there an acceptable way to dump them all and start over /without losing the contents of the IPA database/? Or otherwise really screwing ourselves? I don't believe there is a way. We have a replica that's still up and running and we've switched everyone over to talking to it, but we're at risk with just the one. I'd ignore the two unknown certs for now. They look like someone was experimenting with issuing a cert and didn't quite get things working. The CA seems to be throwing an error. I'd check the syslog for messages from certmonger and look at the CA debug log and selftest log. rob [snip] -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
My system shows pki-server is installed and V10.2.1-3.fc21, but I don't have the pki-server binary itself. Will reinstalling this rpm hurt me in any way? Without it, I'm not sure how to check my system against the messages you provided below. On 04/28/2016 11:07 AM, Petr Vobornik wrote: On 04/28/2016 04:07 PM, Bret Wortman wrote: Okay. This morning, I turned back time to 4/1 and started up IPA. It didn't work, but I got something new and interesting in the debug log, which I've posted to http://pastebin.com/M9VGCS8A. Lots of garbled junk came pouring out which doesn't happen when I'm set to real time. Is /this/ significant? Anything in systemctl status pki-tomcatd@pki-tomcat.service or rather: journalctl -u pki-tomcatd@pki-tomcat.service ? Just to be sure, it might be also worth to check if CA subsystem users have correct certs assigned: * https://www.redhat.com/archives/freeipa-users/2016-April/msg00138.html * https://www.redhat.com/archives/freeipa-users/2016-April/msg00143.html On 04/27/2016 02:24 PM, Bret Wortman wrote: I put excerpts from the ca logs in http://pastebin.com/gYgskU79. It looks logical to me, but I can't spot anything that looks like a root cause error. The selftests are all okay, I think. The debug log might have something, but it might also just be complaining about ldap not being up because it's not. On 04/27/2016 01:11 PM, Rob Crittenden wrote: Bret Wortman wrote: So in lieu of fixing these certs, is there an acceptable way to dump them all and start over /without losing the contents of the IPA database/? Or otherwise really screwing ourselves? I don't believe there is a way. We have a replica that's still up and running and we've switched everyone over to talking to it, but we're at risk with just the one. I'd ignore the two unknown certs for now. They look like someone was experimenting with issuing a cert and didn't quite get things working. The CA seems to be throwing an error. I'd check the syslog for messages from certmonger and look at the CA debug log and selftest log. rob [snip] -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Okay. This morning, I turned back time to 4/1 and started up IPA. It didn't work, but I got something new and interesting in the debug log, which I've posted to http://pastebin.com/M9VGCS8A. Lots of garbled junk came pouring out which doesn't happen when I'm set to real time. Is /this/ significant? On 04/27/2016 02:24 PM, Bret Wortman wrote: I put excerpts from the ca logs in http://pastebin.com/gYgskU79. It looks logical to me, but I can't spot anything that looks like a root cause error. The selftests are all okay, I think. The debug log might have something, but it might also just be complaining about ldap not being up because it's not. On 04/27/2016 01:11 PM, Rob Crittenden wrote: Bret Wortman wrote: So in lieu of fixing these certs, is there an acceptable way to dump them all and start over /without losing the contents of the IPA database/? Or otherwise really screwing ourselves? I don't believe there is a way. We have a replica that's still up and running and we've switched everyone over to talking to it, but we're at risk with just the one. I'd ignore the two unknown certs for now. They look like someone was experimenting with issuing a cert and didn't quite get things working. The CA seems to be throwing an error. I'd check the syslog for messages from certmonger and look at the CA debug log and selftest log. rob [snip] -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
I put excerpts from the ca logs in http://pastebin.com/gYgskU79. It looks logical to me, but I can't spot anything that looks like a root cause error. The selftests are all okay, I think. The debug log might have something, but it might also just be complaining about ldap not being up because it's not. On 04/27/2016 01:11 PM, Rob Crittenden wrote: Bret Wortman wrote: So in lieu of fixing these certs, is there an acceptable way to dump them all and start over /without losing the contents of the IPA database/? Or otherwise really screwing ourselves? I don't believe there is a way. We have a replica that's still up and running and we've switched everyone over to talking to it, but we're at risk with just the one. I'd ignore the two unknown certs for now. They look like someone was experimenting with issuing a cert and didn't quite get things working. The CA seems to be throwing an error. I'd check the syslog for messages from certmonger and look at the CA debug log and selftest log. rob [snip] -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA server having cert issues
Was this at all informative? On 04/26/2016 02:06 PM, Bret Wortman wrote: On 04/26/2016 01:45 PM, Rob Crittenden wrote: Bret Wortman wrote: I think I've found a deeper problem, in that I can't update these because IPA simply won't start at all now. I mistyped one of these -- the 2016-03-11 is actually 2018-03-11, and 2016-04-01 is actually 2036-04-01. As for the unknowns, the first says status: CA_REJECTED and the error says "hostname in subject of request 'zw198.private.net' does not match principal hostname 'private.net'", with stuck: yes. The second is similar, but for a different host. Is it really a different host and why? I think we'd need to see the full output to know what's going on. Full output: Number of certificates and requests being tracked: 10. Request ID '20140428181940': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-NET',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-PRIVATE-NET/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-NET',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=zsipa.private.net,O=PRIVATE.NET expires: 2018-04-02 13:04:51 UTC principal name: ldap/zsipa.private@private.net key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: track: yes auto-renew: yes Request ID '20140428182016': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=zsipa.private.net,O=PRIVATE.NET expires: 2018-04-02 13:04:31 UTC principal name: HTTP/zsipa.private@private.net key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150211141945': status: CA_REJECTED ca-error: Server at https://zsipa.private.net/ipa/xml denied our request, giving up: 2100 (RPC failed at server. Insufficient access: hostname in subject of request 'zw198.private.net' does not match principal hostname 'private.net'). stuck: yes key pair storage: type=NSSDB,location='/etc/pki/nssdb',nickname='Server-Cert',token='NSS Certificate DB' certificate: type=NSSDB,location='/etc/pki/nssdb',nickname='Server-Cert' CA: IPA issuer: subject: expires: unknown pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150816194107': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='424151811070' certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=CA Audit,O=PRIVATE.NET expires: 2016-04-17 18:19:19 UTC key usage: digitalSignature,nonRepudiation pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150816194108': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin='424151811070' certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=OCSP Subsystem,O=PRIVATE.NET expires: 2016-04-17 18:19:18 UTC key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign eku: id-kp-OCSPSigning pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150816194109': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin='424151811070' certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=CA Subsystem,O=PRIVATE.NET expires: 2016-04-17 18:19:19 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
Re: [Freeipa-users] IPA server having cert issues
On 04/26/2016 01:45 PM, Rob Crittenden wrote: Bret Wortman wrote: I think I've found a deeper problem, in that I can't update these because IPA simply won't start at all now. I mistyped one of these -- the 2016-03-11 is actually 2018-03-11, and 2016-04-01 is actually 2036-04-01. As for the unknowns, the first says status: CA_REJECTED and the error says "hostname in subject of request 'zw198.private.net' does not match principal hostname 'private.net'", with stuck: yes. The second is similar, but for a different host. Is it really a different host and why? I think we'd need to see the full output to know what's going on. Full output: Number of certificates and requests being tracked: 10. Request ID '20140428181940': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-NET',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-PRIVATE-NET/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-NET',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=zsipa.private.net,O=PRIVATE.NET expires: 2018-04-02 13:04:51 UTC principal name: ldap/zsipa.private@private.net key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: track: yes auto-renew: yes Request ID '20140428182016': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=zsipa.private.net,O=PRIVATE.NET expires: 2018-04-02 13:04:31 UTC principal name: HTTP/zsipa.private@private.net key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150211141945': status: CA_REJECTED ca-error: Server at https://zsipa.private.net/ipa/xml denied our request, giving up: 2100 (RPC failed at server. Insufficient access: hostname in subject of request 'zw198.private.net' does not match principal hostname 'private.net'). stuck: yes key pair storage: type=NSSDB,location='/etc/pki/nssdb',nickname='Server-Cert',token='NSS Certificate DB' certificate: type=NSSDB,location='/etc/pki/nssdb',nickname='Server-Cert' CA: IPA issuer: subject: expires: unknown pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150816194107': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin='424151811070' certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=CA Audit,O=PRIVATE.NET expires: 2016-04-17 18:19:19 UTC key usage: digitalSignature,nonRepudiation pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150816194108': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin='424151811070' certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=OCSP Subsystem,O=PRIVATE.NET expires: 2016-04-17 18:19:18 UTC key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign eku: id-kp-OCSPSigning pre-save command: post-save command: track: yes auto-renew: yes Request ID '20150816194109': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin='424151811070' certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=PRIVATE.NET subject: CN=CA Subsystem,O=PRIVATE.NET expires: 2016-04-17 18:19:19 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-sa
Re: [Freeipa-users] IPA server having cert issues
I should also note that /var/log/dirsrv/slapd-PRIVATE-NET/errors ends with a series of "csngen_new_csn - Warning: too much time skew (-2153860 secs). Current seqnum=1" errors. On 04/26/2016 12:57 PM, Bret Wortman wrote: I think I've found a deeper problem, in that I can't update these because IPA simply won't start at all now. I mistyped one of these -- the 2016-03-11 is actually 2018-03-11, and 2016-04-01 is actually 2036-04-01. As for the unknowns, the first says status: CA_REJECTED and the error says "hostname in subject of request 'zw198.private.net' does not match principal hostname 'private.net'", with stuck: yes. The second is similar, but for a different host. No idea what's wrong with the rest, or why nothing will start. Near as I can tell, Kerberos is failing to start, which is causing everything else to go toes up. Early in the startup, in /var/log/messages, there's: ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No Kerberos credentials available) After that, I get a jar file read pboelm on log4j.jar, then a series of property setting attempts that don't find matching properties. Then some cipher errors, then it looks like named starts up okay, and everything pauses for about 5 minutes before it all comes crashing back down. Bret On 04/26/2016 12:40 PM, Petr Vobornik wrote: On 04/26/2016 06:00 PM, Bret Wortman wrote: # getcert list | grep expires expires: 2018-04-02 13:04:51 UTC expires: 2018-04-02 13:04:31 UTC expires: unknown expires: 2016-04-17 18:19:19 UTC expires: 2016-04-17 18:19:18 UTC expires: 2016-04-17 18:19:19 UTC expires: 2016-04-01 20:16:39 UTC expires: 2016-04-17 18:19:35 UTC expires: 2016-03-11 13:04:29 UTC expires: unknown # So some got updated and most didn't. Is there a recommended way to update these all? The system is still backdated to 3 April (ntpd disabled) at this point. It's usually good to start renewing(when it doesn't happen automatically from some reason) with the cert which is about to expired first, i.e. the one with "2016-03-11 13:04:29" The process is: - move date before the cert is about to expired - leave it up to certmonger or manually force resubmit by `getcert resubmit -i $REQUEST_ID`, where request ID is in `getcert list` output. I'm little worried about the fact that CA cert was renewed at date which is after expiration of the other certs. Also the `expires: unknown` doesn't look good. Check `getcert list` output for errors related to the cert. Bret On 04/26/2016 11:46 AM, Petr Vobornik wrote: On 04/26/2016 03:26 PM, Bret Wortman wrote: On our non-CA IPA server, this is happening, in case it's related and illustrative: # ipa host-del zw113.private.net ipa: ERROR: Certificate format error: (SEC_ERROR_LEGACY_DATABASE) The certificate/key database is in an old, unsupported format. # I would start with checking on all IPA servers if and what certificates are expired: # getcert list or short version to check if there are any: # getcert list | grep expires When CA cert is renewed, it is not automatically transfered to clients. There one must run: # ipa-certupdate On 04/26/2016 09:24 AM, Bret Wortman wrote: I rolled the date on the IPA server in question back to April 1 and ran "ipa-cacert-manage renew", which said it completed successfully. I rolled the date back to current and tried restarting ipa using ipactl stop && ipactl start, but no joy. No more ca renewal errors, but right after the pause I see this in /var/log/messages: systemd: kadmin.service: main process exited, code=exited, status=2/INVALIDARGUMENT systemd: Unit kadmin.service entered failed state. systemd: kadmin.service failed. I rebooted the server just in case, and it's still getting stuck at the same place. ipa-otpd doesn't get around to starting. Bret After the several-minutes-long pause after ipactl start outputs "Starting pki-tomcatd Service", I get the On 04/26/2016 08:14 AM, Bret Wortman wrote: I have an IPA server on a private network which has apparently run into certificate issues this morning. It's been running without issue for quite a while, and is on 4.1.4-1 on fedora 21. This morning, the gui started giving: IPA Error 907: NetworkError with description "cannot connect to 'https://zsipa.private.net:443/ca/agent/ca/displayBySerial': (SSL_ERROR_EXPIRED_CERRT_ALERT) SSL peer rejected your certificate as expired." I dug into the logs and after trying to restart ipa using ipactl, there was a length pause, then: dogtag-ipa-ca-renew-agent-submit: Updated certificate not available certmonger: Certificate named "ipaCert" in token "NSS Certificate DB" in database "/etc/httpd/alias" is no longer valid. dogtag-ipa-ca-renew-agent-submit: Updated certificate not available certmonger: Certificate named "ocspSigningCer
Re: [Freeipa-users] IPA server having cert issues
I think I've found a deeper problem, in that I can't update these because IPA simply won't start at all now. I mistyped one of these -- the 2016-03-11 is actually 2018-03-11, and 2016-04-01 is actually 2036-04-01. As for the unknowns, the first says status: CA_REJECTED and the error says "hostname in subject of request 'zw198.private.net' does not match principal hostname 'private.net'", with stuck: yes. The second is similar, but for a different host. No idea what's wrong with the rest, or why nothing will start. Near as I can tell, Kerberos is failing to start, which is causing everything else to go toes up. Early in the startup, in /var/log/messages, there's: ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No Kerberos credentials available) After that, I get a jar file read pboelm on log4j.jar, then a series of property setting attempts that don't find matching properties. Then some cipher errors, then it looks like named starts up okay, and everything pauses for about 5 minutes before it all comes crashing back down. Bret On 04/26/2016 12:40 PM, Petr Vobornik wrote: On 04/26/2016 06:00 PM, Bret Wortman wrote: # getcert list | grep expires expires: 2018-04-02 13:04:51 UTC expires: 2018-04-02 13:04:31 UTC expires: unknown expires: 2016-04-17 18:19:19 UTC expires: 2016-04-17 18:19:18 UTC expires: 2016-04-17 18:19:19 UTC expires: 2016-04-01 20:16:39 UTC expires: 2016-04-17 18:19:35 UTC expires: 2016-03-11 13:04:29 UTC expires: unknown # So some got updated and most didn't. Is there a recommended way to update these all? The system is still backdated to 3 April (ntpd disabled) at this point. It's usually good to start renewing(when it doesn't happen automatically from some reason) with the cert which is about to expired first, i.e. the one with "2016-03-11 13:04:29" The process is: - move date before the cert is about to expired - leave it up to certmonger or manually force resubmit by `getcert resubmit -i $REQUEST_ID`, where request ID is in `getcert list` output. I'm little worried about the fact that CA cert was renewed at date which is after expiration of the other certs. Also the `expires: unknown` doesn't look good. Check `getcert list` output for errors related to the cert. Bret On 04/26/2016 11:46 AM, Petr Vobornik wrote: On 04/26/2016 03:26 PM, Bret Wortman wrote: On our non-CA IPA server, this is happening, in case it's related and illustrative: # ipa host-del zw113.private.net ipa: ERROR: Certificate format error: (SEC_ERROR_LEGACY_DATABASE) The certificate/key database is in an old, unsupported format. # I would start with checking on all IPA servers if and what certificates are expired: # getcert list or short version to check if there are any: # getcert list | grep expires When CA cert is renewed, it is not automatically transfered to clients. There one must run: # ipa-certupdate On 04/26/2016 09:24 AM, Bret Wortman wrote: I rolled the date on the IPA server in question back to April 1 and ran "ipa-cacert-manage renew", which said it completed successfully. I rolled the date back to current and tried restarting ipa using ipactl stop && ipactl start, but no joy. No more ca renewal errors, but right after the pause I see this in /var/log/messages: systemd: kadmin.service: main process exited, code=exited, status=2/INVALIDARGUMENT systemd: Unit kadmin.service entered failed state. systemd: kadmin.service failed. I rebooted the server just in case, and it's still getting stuck at the same place. ipa-otpd doesn't get around to starting. Bret After the several-minutes-long pause after ipactl start outputs "Starting pki-tomcatd Service", I get the On 04/26/2016 08:14 AM, Bret Wortman wrote: I have an IPA server on a private network which has apparently run into certificate issues this morning. It's been running without issue for quite a while, and is on 4.1.4-1 on fedora 21. This morning, the gui started giving: IPA Error 907: NetworkError with description "cannot connect to 'https://zsipa.private.net:443/ca/agent/ca/displayBySerial': (SSL_ERROR_EXPIRED_CERRT_ALERT) SSL peer rejected your certificate as expired." I dug into the logs and after trying to restart ipa using ipactl, there was a length pause, then: dogtag-ipa-ca-renew-agent-submit: Updated certificate not available certmonger: Certificate named "ipaCert" in token "NSS Certificate DB" in database "/etc/httpd/alias" is no longer valid. dogtag-ipa-ca-renew-agent-submit: Updated certificate not available certmonger: Certificate named "ocspSigningCert cert-pki-ca" in token "NSS Certificate DB" in database "/etc/pki/pki-tomcat/alias" is no longer valid. dogtag-ipa-ca-renew-agent-submit: Updated certificate not available. named-pkcs11[3437]: client 192.168.208
Re: [Freeipa-users] IPA server having cert issues
# getcert list | grep expires expires: 2018-04-02 13:04:51 UTC expires: 2018-04-02 13:04:31 UTC expires: unknown expires: 2016-04-17 18:19:19 UTC expires: 2016-04-17 18:19:18 UTC expires: 2016-04-17 18:19:19 UTC expires: 2016-04-01 20:16:39 UTC expires: 2016-04-17 18:19:35 UTC expires: 2016-03-11 13:04:29 UTC expires: unknown # So some got updated and most didn't. Is there a recommended way to update these all? The system is still backdated to 3 April (ntpd disabled) at this point. Bret On 04/26/2016 11:46 AM, Petr Vobornik wrote: On 04/26/2016 03:26 PM, Bret Wortman wrote: On our non-CA IPA server, this is happening, in case it's related and illustrative: # ipa host-del zw113.private.net ipa: ERROR: Certificate format error: (SEC_ERROR_LEGACY_DATABASE) The certificate/key database is in an old, unsupported format. # I would start with checking on all IPA servers if and what certificates are expired: # getcert list or short version to check if there are any: # getcert list | grep expires When CA cert is renewed, it is not automatically transfered to clients. There one must run: # ipa-certupdate On 04/26/2016 09:24 AM, Bret Wortman wrote: I rolled the date on the IPA server in question back to April 1 and ran "ipa-cacert-manage renew", which said it completed successfully. I rolled the date back to current and tried restarting ipa using ipactl stop && ipactl start, but no joy. No more ca renewal errors, but right after the pause I see this in /var/log/messages: systemd: kadmin.service: main process exited, code=exited, status=2/INVALIDARGUMENT systemd: Unit kadmin.service entered failed state. systemd: kadmin.service failed. I rebooted the server just in case, and it's still getting stuck at the same place. ipa-otpd doesn't get around to starting. Bret After the several-minutes-long pause after ipactl start outputs "Starting pki-tomcatd Service", I get the On 04/26/2016 08:14 AM, Bret Wortman wrote: I have an IPA server on a private network which has apparently run into certificate issues this morning. It's been running without issue for quite a while, and is on 4.1.4-1 on fedora 21. This morning, the gui started giving: IPA Error 907: NetworkError with description "cannot connect to 'https://zsipa.private.net:443/ca/agent/ca/displayBySerial': (SSL_ERROR_EXPIRED_CERRT_ALERT) SSL peer rejected your certificate as expired." I dug into the logs and after trying to restart ipa using ipactl, there was a length pause, then: dogtag-ipa-ca-renew-agent-submit: Updated certificate not available certmonger: Certificate named "ipaCert" in token "NSS Certificate DB" in database "/etc/httpd/alias" is no longer valid. dogtag-ipa-ca-renew-agent-submit: Updated certificate not available certmonger: Certificate named "ocspSigningCert cert-pki-ca" in token "NSS Certificate DB" in database "/etc/pki/pki-tomcat/alias" is no longer valid. dogtag-ipa-ca-renew-agent-submit: Updated certificate not available. named-pkcs11[3437]: client 192.168.208.205#57832: update '208.168.192.in-addr.arpa/IN' denied and then things start shutting down. I can't start ipa at all using ipactl. So at present, our DNS is down. Authentication should work for a while, but I'd like to get this working again as quickly as possible. Any ideas? I deal with certificates so infrequently (like only when something like this happens) that I'm not sure where to start. Thanks! -- *Bret Wortman* /Coming soon to Kickstarter.../ <http://wrapbuddies.co/> http://wrapbuddies.co/ -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] DNS not responding properly....
Check. [root@ipa1 data]# ipa dnszone-show foo.net Zone name: foo.net Authoritative nameserver: ipa1.foo.net. Administrator e-mail address: hostmaster.foo.net. SOA serial: 1400521450 SOA refresh: 3600 SOA retry: 900 SOA expire: 1209600 SOA minimum: 3600 Active zone: TRUE Allow query: any; Allow transfer: none; Zone forwarders: 8.8.8.8 [root@ipa1 data]# On 09/05/2014 01:56 PM, Petr Spacek wrote: Hello, On 5.9.2014 18:14, Bret Wortman wrote: I've got an odd situation with one of our networks. Our systems are properly registered in DNS within IPA, and the web interface and IPA queries work to resolve the hosts, but named isn't playing along with us. [root@ipa1 data]# ipa dnsrecord-find foo.net --name=asterisk Record name: asterisk A record: 192.168.252.155 Number of entries returned 1 [root@ipa1 data]# host asterisk.foo.net Host asterisk.foo.net not found: 3(NXDOMAIN) [root@ipa1 data]# cat /etc/resolv.conf search foo.net nameserver 192.168.252.61- This is ipa1 nameserver 192.168.252.62 nameserver 192.168.252.63 [root@ipa1 data]# ifconfig ens192: flags=4163UP,BROADCAST,RUNNING,MULTICAST mtu 1500 inet 192.168.252.61 netmask 255.255.255.0 broadcast 192.168.252.255 inet6 fe80::250:56ff:fe04:401 prefixlen 64 scopeid 0x20link ether 00:50:56:04:04:01 txqueuelen 1000 (Ethernet) RX packets 2189 bytes 332143 (324.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1523 bytes 428925 (418.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73UP,LOOPBACK,RUNNING mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10host loop txqueuelen 0 (Local Loopback) RX packets 1037 bytes 718872 (702.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1037 bytes 718872 (702.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@ipa1 data]# When I dig into the named.run file, I see the trace below (I ran an rndc reload after seeing the request to do so at the end of an earlier section of the file; it obviously didn't help much). I'm not sure where else to look. /etc/named.conf and /var/named/named.ca both are in line with what we have on another similar system where everything is working just fine. Any thoughts? Please double check output from $ ipa dnszone-show foo.net It should contain line like: Active zone: TRUE Petr^2 Spacek 05-Sep-2014 12:04:47.111 received control channel command 'reload' 05-Sep-2014 12:04:47.111 zone 252.168.192.in-addr.arpa/IN: shutting down 05-Sep-2014 12:04:47.112 loading configuration from '/etc/named.conf' 05-Sep-2014 12:04:47.112 using default UDP/IPv4 port range: [1024, 65535] 05-Sep-2014 12:04:47.112 using default UDP/IPv6 port range: [1024, 65535] 05-Sep-2014 12:04:47.113 sizing zone task pool based on 6 zones 05-Sep-2014 12:04:47.116 option 'serial_autoincrement' is not supported, ignoring 05-Sep-2014 12:04:47.194 automatic empty zone: 10.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 16.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 17.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 18.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 19.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 20.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 21.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 22.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 23.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 24.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.194 automatic empty zone: 25.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.195 automatic empty zone: 26.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 27.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 28.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 29.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 30.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 31.172.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 168.192.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 64.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 65.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.196 automatic empty zone: 66.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 67.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 68.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 69.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 70.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 71.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 72.100.IN-ADDR.ARPA 05-Sep-2014 12:04:47.198 automatic empty zone: 73.100.IN-ADDR.ARPA 05
[Freeipa-users] DNS not responding properly....
,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.318 update_record (syncrepl) failed, dn 'idnsname=_kerberos-master._tcp,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.318 update_record (syncrepl) failed, dn 'idnsname=_kerberos-master._udp,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.318 update_record (syncrepl) failed, dn 'idnsname=_kpasswd._tcp,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.318 update_record (syncrepl) failed, dn 'idnsname=_kpasswd._udp,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.319 update_record (syncrepl) failed, dn 'idnsname=_ntp._udp,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.319 update_record (syncrepl) failed, dn 'idnsname=ipa-ca,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.319 update_record (syncrepl) failed, dn 'idnsname=ipa2,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.319 update_record (syncrepl) failed, dn 'idnsname=mcnetmon,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.320 update_record (syncrepl) failed, dn 'idnsname=asterisk,idnsname=foo.net,cn=dns,dc=foo,dc=net' change type 0x1. Records can be outdated, run `rndc reload`: not found 05-Sep-2014 12:04:47.320 zone 252.168.192.in-addr.arpa/IN: loaded serial 1409933087 05-Sep-2014 12:04:47.320 1 zones from LDAP instance 'ipa' loaded (1 zones defined) -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
[Freeipa-users] GSSAPIAuthentication setting in /etc/sshd_config?
Does this really need to be set to yes in /etc/sshd_config? I've looked through the documentation and it only seems to say this for HP-UX and AIX. We're running freeipa 3.3.5-1 and are seeing some slow logins via ssh that some users have reported speed up markedly when this setting is toggled to no. Before I make any wholesale change recommendations, I wanted to check on this. Thanks! -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] LDAP/SSSD/IPA performance
The CD is in the hands of the security folks now. I'll let you know when I have it and can transfer the logs over to you. It's only 2GB worth of data, but I hope it's informative. Bret On 05/28/2014 03:52 AM, Jakub Hrozek wrote: On Tue, May 27, 2014 at 07:34:58PM -0400, Bret Wortman wrote: No problem. We forced a re installation of openldap, which helped. Pam login is still slow but sudo isn't. We'll keep chipping away at it. As said earlier in the thread, logs might be the best way to move this forward. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] LDAP/SSSD/IPA performance
I'll get with my network guys and start troubleshooting. Thanks! On 05/27/2014 09:20 AM, Dmitri Pal wrote: On 05/27/2014 08:41 AM, Rob Crittenden wrote: Bret Wortman wrote: Crud. That was supposed to have a second comparison log too: I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT err=0 tag=97 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=2 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(cn=deraults) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=3 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(|(sudoUser=bretw)(sudoUser=%users)(sudoUser=%#100)(sudoUser=%admins)(sudoUser=%nonexp)(sudoUser=%sudoers)(sudoUser=$unrestricted)(sudoUser=%#185520)(sudoUser=%#1855204) (sudoUser=%#185526)(sudoUser=%#185527)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=3 RESULT erro=0 tag=101 nentries=2 etime=0 [26/May/2014:09:09:01 -0400] conn=183751 op=4 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:09:01 -0400] conn=183751op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:03 -0400] conn=183751 op=5 UNBIND [26/May/2014:09:09:03 -0400] conn=183751 op=5 fd=111 closed = U1 I think this shows, roughly, a 7 second elapsed time from start to finish, right? Granted, there were other request being serficed during this interval as well, but nothing that looked like outrageous volume. I don't see anything unusual here. The directory server retrieved the data just as fast on both systems, the difference appears to be the network, in connection and shutdown times. +1, however the TLS handshake took longer. That probably required several DNS lookups so I wonder if DNS lookups might be slowing things down. I wonder if putting server records manually into the hosts file would make a difference. If yes then may be you need to take a look at your DNS setup for the slow network. On our faster network, this same exchange went much faster: [26/May/2014:09:22:55 -0400] conn=12896 fd=100 slot=100 connection from 192.168.2.13 to 192.168.2.61 [26/May/2014:09:22:55 -0400] conn=12896 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:22:55 -0400] conn=12896 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 SSL 128-bit AES [26/May/2014:09:22:56 -0400] conn=12896 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me method=128 version=3 [26/May/2014:09:22:56 -0400] conn=12896 op=1 RESULT err=0 tag=97 nentries=0 etime=0 dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me [26/May/2014:09:22:56 -0400] conn=12896 op=2 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(cn=defaults) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=3 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(|(sudoUser=bretw)(sudoUser=%bretw)(sudoUser=%#10042)(sudoUser=%admins)(sudoUser=%#38880)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=3 RESULT err=0 tag=101 nentries=1 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=4 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=5 UNBIND [26/May/2014:09:22:56 -0400] conn=12896 op=5 fd=100 closed - U1 Bret On 05/26/2014 09:51 AM, Bret Wortman wrote: Okay, I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT err=0 tag=97 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=2 SRCH base=ou
Re: [Freeipa-users] LDAP/SSSD/IPA performance
I just checked to be sure, and we do already put all the IPA servers in our client host tables just to be sure they can be reached even if DNS goes down. On 05/27/2014 09:20 AM, Dmitri Pal wrote: On 05/27/2014 08:41 AM, Rob Crittenden wrote: Bret Wortman wrote: Crud. That was supposed to have a second comparison log too: I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT err=0 tag=97 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=2 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(cn=deraults) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=3 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(|(sudoUser=bretw)(sudoUser=%users)(sudoUser=%#100)(sudoUser=%admins)(sudoUser=%nonexp)(sudoUser=%sudoers)(sudoUser=$unrestricted)(sudoUser=%#185520)(sudoUser=%#1855204) (sudoUser=%#185526)(sudoUser=%#185527)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=3 RESULT erro=0 tag=101 nentries=2 etime=0 [26/May/2014:09:09:01 -0400] conn=183751 op=4 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:09:01 -0400] conn=183751op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:03 -0400] conn=183751 op=5 UNBIND [26/May/2014:09:09:03 -0400] conn=183751 op=5 fd=111 closed = U1 I think this shows, roughly, a 7 second elapsed time from start to finish, right? Granted, there were other request being serficed during this interval as well, but nothing that looked like outrageous volume. I don't see anything unusual here. The directory server retrieved the data just as fast on both systems, the difference appears to be the network, in connection and shutdown times. +1, however the TLS handshake took longer. That probably required several DNS lookups so I wonder if DNS lookups might be slowing things down. I wonder if putting server records manually into the hosts file would make a difference. If yes then may be you need to take a look at your DNS setup for the slow network. On our faster network, this same exchange went much faster: [26/May/2014:09:22:55 -0400] conn=12896 fd=100 slot=100 connection from 192.168.2.13 to 192.168.2.61 [26/May/2014:09:22:55 -0400] conn=12896 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:22:55 -0400] conn=12896 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 SSL 128-bit AES [26/May/2014:09:22:56 -0400] conn=12896 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me method=128 version=3 [26/May/2014:09:22:56 -0400] conn=12896 op=1 RESULT err=0 tag=97 nentries=0 etime=0 dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me [26/May/2014:09:22:56 -0400] conn=12896 op=2 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(cn=defaults) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=3 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(|(sudoUser=bretw)(sudoUser=%bretw)(sudoUser=%#10042)(sudoUser=%admins)(sudoUser=%#38880)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=3 RESULT err=0 tag=101 nentries=1 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=4 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=5 UNBIND [26/May/2014:09:22:56 -0400] conn=12896 op=5 fd=100 closed - U1 Bret On 05/26/2014 09:51 AM, Bret Wortman wrote: Okay, I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT
Re: [Freeipa-users] LDAP/SSSD/IPA performance
No problem. We forced a re installation of openldap, which helped. Pam login is still slow but sudo isn't. We'll keep chipping away at it. Bret Wortman http://bretwortman.com/ http://twitter.com/BretWortman On May 27, 2014, at 7:15 PM, Dmitri Pal d...@redhat.com wrote: On 05/27/2014 09:44 AM, Bret Wortman wrote: I just checked to be sure, and we do already put all the IPA servers in our client host tables just to be sure they can be reached even if DNS goes down. Sorry, I am running out of ideas. On 05/27/2014 09:20 AM, Dmitri Pal wrote: On 05/27/2014 08:41 AM, Rob Crittenden wrote: Bret Wortman wrote: Crud. That was supposed to have a second comparison log too: I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT err=0 tag=97 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=2 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(cn=deraults) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=3 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(|(sudoUser=bretw)(sudoUser=%users)(sudoUser=%#100)(sudoUser=%admins)(sudoUser=%nonexp)(sudoUser=%sudoers)(sudoUser=$unrestricted)(sudoUser=%#185520)(sudoUser=%#1855204) (sudoUser=%#185526)(sudoUser=%#185527)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=3 RESULT erro=0 tag=101 nentries=2 etime=0 [26/May/2014:09:09:01 -0400] conn=183751 op=4 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:09:01 -0400] conn=183751op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:03 -0400] conn=183751 op=5 UNBIND [26/May/2014:09:09:03 -0400] conn=183751 op=5 fd=111 closed = U1 I think this shows, roughly, a 7 second elapsed time from start to finish, right? Granted, there were other request being serficed during this interval as well, but nothing that looked like outrageous volume. I don't see anything unusual here. The directory server retrieved the data just as fast on both systems, the difference appears to be the network, in connection and shutdown times. +1, however the TLS handshake took longer. That probably required several DNS lookups so I wonder if DNS lookups might be slowing things down. I wonder if putting server records manually into the hosts file would make a difference. If yes then may be you need to take a look at your DNS setup for the slow network. On our faster network, this same exchange went much faster: [26/May/2014:09:22:55 -0400] conn=12896 fd=100 slot=100 connection from 192.168.2.13 to 192.168.2.61 [26/May/2014:09:22:55 -0400] conn=12896 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:22:55 -0400] conn=12896 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 SSL 128-bit AES [26/May/2014:09:22:56 -0400] conn=12896 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me method=128 version=3 [26/May/2014:09:22:56 -0400] conn=12896 op=1 RESULT err=0 tag=97 nentries=0 etime=0 dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me [26/May/2014:09:22:56 -0400] conn=12896 op=2 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(cn=defaults) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=3 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(|(sudoUser=bretw)(sudoUser=%bretw)(sudoUser=%#10042)(sudoUser=%admins)(sudoUser=%#38880)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=3 RESULT err=0 tag=101 nentries=1 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=4 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=5 UNBIND [26/May/2014:09:22:56 -0400] conn=12896 op=5 fd=100 closed - U1 Bret On 05/26/2014 09:51 AM, Bret Wortman wrote: Okay, I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26
Re: [Freeipa-users] LDAP/SSSD/IPA performance
Crud. That was supposed to have a second comparison log too: I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT err=0 tag=97 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=2 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(cn=deraults) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=3 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(|(sudoUser=bretw)(sudoUser=%users)(sudoUser=%#100)(sudoUser=%admins)(sudoUser=%nonexp)(sudoUser=%sudoers)(sudoUser=$unrestricted)(sudoUser=%#185520)(sudoUser=%#1855204) (sudoUser=%#185526)(sudoUser=%#185527)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=3 RESULT erro=0 tag=101 nentries=2 etime=0 [26/May/2014:09:09:01 -0400] conn=183751 op=4 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:09:01 -0400] conn=183751op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:03 -0400] conn=183751 op=5 UNBIND [26/May/2014:09:09:03 -0400] conn=183751 op=5 fd=111 closed = U1 I think this shows, roughly, a 7 second elapsed time from start to finish, right? Granted, there were other request being serficed during this interval as well, but nothing that looked like outrageous volume. On our faster network, this same exchange went much faster: [26/May/2014:09:22:55 -0400] conn=12896 fd=100 slot=100 connection from 192.168.2.13 to 192.168.2.61 [26/May/2014:09:22:55 -0400] conn=12896 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:22:55 -0400] conn=12896 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 SSL 128-bit AES [26/May/2014:09:22:56 -0400] conn=12896 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me method=128 version=3 [26/May/2014:09:22:56 -0400] conn=12896 op=1 RESULT err=0 tag=97 nentries=0 etime=0 dn=uid=sudo,cn=sysaccounts,cn=etc,dc=wedgeofli,dc=me [26/May/2014:09:22:56 -0400] conn=12896 op=2 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(cn=defaults) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=3 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(|(sudoUser=bretw)(sudoUser=%bretw)(sudoUser=%#10042)(sudoUser=%admins)(sudoUser=%#38880)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=3 RESULT err=0 tag=101 nentries=1 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=4 SRCH base=ou=SUDOers,dc=wedgeofli,dc=me scope=2 filter=(sudoUser=+*) attrs=ALL [26/May/2014:09:22:56 -0400] conn=12896 op=4 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:22:56 -0400] conn=12896 op=5 UNBIND [26/May/2014:09:22:56 -0400] conn=12896 op=5 fd=100 closed - U1 Bret On 05/26/2014 09:51 AM, Bret Wortman wrote: Okay, I found something in the slapd-FOO-NET/access log. I figured out which conn ID related to a sudo -i that I performed which took longer than expected and grepped for that conn ID: [26/May/2014:09:08:56 -0400] conn=183751 fd=111 slot=111 connection from 192.168.208.129 to 192.168.10.111 [26/May/2014:09:08:57 -0400] conn=183751 op=0 EXT oid=1.3.6.1.4.1.1466.20037 name=startTLS [26/May/2014:09:08:57 -0400] conn=183751 op=0 RESULT err=0 tag=120 nentries=0 etime=0 [26/May/2014:09:08:59 -0400] conn=183751 SSL 128-bit AES [26/May/2014:09:08:59 -0400] conn=183751 op=1 BIND dn=uid=sudo,cn=sysaccounts,cn=etc,dc=foo,dc=net method=128 version=3 [26/May/2014:09:08:59 -0400] conn=183751 op=1 RESULT err=0 tag=97 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=2 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(cn=deraults) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=2 RESULT err=0 tag=101 nentries=0 etime=0 [26/May/2014:09:09:00 -0400] conn=183751 op=3 SRCH base=ou=SUDOers,dc=foo,dc=net scope=2 filter=(|(sudoUser=bretw)(sudoUser=%users)(sudoUser=%#100)(sudoUser=%admins)(sudoUser=%nonexp)(sudoUser=%sudoers)(sudoUser=$unrestricted)(sudoUser=%#185520)(sudoUser=%#1855204) (sudoUser=%#185526)(sudoUser=%#185527)(sudoUser=ALL)) attrs=ALL [26/May/2014:09:09:00 -0400] conn=183751 op=3 RESULT erro=0 tag=101 nentries=2 etime=0 [26/May/2014:09:09:01 -0400] conn=183751 op=4 SRCH base=ou=SUDOers
Re: [Freeipa-users] Export user and host list to a csv or text file
Yes, though it might be a bit more data than you're expecting.
Here's what we did to get the details out of a server (and import them
into another). I'm sure there's a more elegant solution, but this worked
for us. Also note that we didn't use all the data this export script
generated, but felt it was better to have it than to not.
EXPORT:
#!/bin/sh
#
# Generate latest ipa config files for possible re-import later.
#
# (C) 2014, The Damascus Group
#
CONFIGDIR=/opt/ipa_config
[ ! -d $CONFIGDIR ] mkdir $CONFIGDIR
pushd $CONFIGDIR
ipa dnszone-find --all dnszone.txt
grep 'Zone name' dnszone.txt | awk '{print $3}' | sed 's/\r//' zones.txt
for line in $(cat zones.txt); do
fn=$(echo $line | sed 's/\.in-addr\.arpa\.//')
echo For zone $line - dnsrecord-$fn.txt
ipa dnsrecord-find $line --sizelimit=9 --all --structured
dnsrecord-${fn}.txt
done
ipa user-find --all users.txt
ipa host-find --sizelimit=9 --all hosts.txt
ipa policy-find --all policy.txt
ipa sudorule-find --all sudorule.txt
ipa sudocmdgroup-find --all sudocmdgroup.txt
ipa sudocmd-find --all sudocmd.txt
ipa role-find --all roles.txt
ipa pwpolicy-find --all pwpolicy.txt
ipa privilege-find --all privilege.txt
ipa permission-find --all permission.txt
ipa netgroup-find --all netgroup.txt
ipa usergroup-find --all usergroup.txt
ipa idrange-find --all idrange.txt
ipa hostgroup-find --all hostgroup.txt
ipahbacrule-find --all hbacrule.txt
ipa hbacsvc-find --all hbacsvc.txt
ipa group-find --all group.txt
ipa cert-find --all cert.txt
ipa automember-find --type=group --all automember-group.txt
ipa automember-find --type=hostgroup --all automember-hostgroup.txt
popd
--cut---
Then, for example, you can import these into a new IPA server using
something like these:
#!/bin/bash
#
# parse_hosts
#
# (C) 2014, The Damascus Group
#
FN=$1
OTP=MyOnetimePassword
RE_HOSTNAME=Host name:\s+(.*)$
name=
while read line; do
if [[ $line =~ $name ]]; then
if [[ -n $name ]]; then
echo Adding $name
ipa host-add $name --password $OTP --force
fi
name=${BASH_REMATCH[1]}
fi
done $FN
echo Adding $name
ipa host-add $name --password $OTP --force
---cut--
And this for users:
#!/bin/bash
#
# parse_users
#
# (C) 2014, The Damascus Group
FN=$1
RE_DN=dn:\s+(.*)$
RE_LOGIN=User login:\s+(.*)$
RE_LAST=Last name:\s+(.*)$
RE_FIRST=First name:\s+(.*)$
RE_CN=Full name:\s+(.*)$
RE_DISPLAYNAME=Display name:\s+(.*)$
RE_INITIALS=Initials:\s+(.*)$
RE_SHELL=Login shell:\s+(.*)$
RE_HOMEDIR=Home directory:\s+(.*)$
RE_PRINCIPAL=Kerberos principal:\s+(.*)$
RE_EMAIL=Email address:\s+(.*)$
RE_SSHPUBKEY=SSH public key:\s+(.*)$
RE_UID=UID:\s+(.*)$
RE_GID=GID:\s+(.*)$
login=
last=
first=
cn=
displayname=
initials=
shell=
homedir=
prinicpal=
email=
sshpubkey=
uid=
gid=
while read line; do
if [[ $line =~ $RE_DN ]]; then
ipa user-add $login \
--last=$last \
--first=$first \
--cn=$cn \
--displayname=$displayname \
--initials=$initials \
--shell=$shell \
--homedir=$homedir \
--principal=$principal \
--email=$email \
--sshpubkey=$sshpubkey \
--uid=$uid \
--gid=$gid
login=
last=
first=
cn=
displayname=
initials=
shell=
homedir=
prinicpal=
email=
sshpubkey=
uid=
gid=
fi
if [[ $line =~ $RE_LOGIN ]]; then
login=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_LAST ]]; then
last=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_FIRST ]]; then
first=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_CN ]]; then
cn=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_DISPLAYNAME ]]; then
displayname=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_INITIALS ]]; then
initials=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_SHELL ]]; then
shell=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_HOMEDIR ]]; then
homedir=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_PRINCIPAL ]]; then
principal=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_EMAIL ]]; then
email=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_SSHPUBKEY ]]; then
sshpubkey1=${BASH_REMATCH[1]}
read sshpubkey2
read sshpubkey3
sshpubkey=$sshpubkey1 $sshpubkey2 $sshpubkey3
fi
if [[ $line =~ $RE_UID ]]; then
uid=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_GID ]]; then
gid=${BASH_REMATCH[1]}
fi
done $FN
ipa user-add $login \
--last=$last \
--first=$first \
--cn=$cn \
--displayname=$displayname \
--initials=$initials \
--shell=$shell \
--homedir=$homedir \
--principal=$principal \
--email=$email \
--sshpubkey=$sshpubkey \
--uid=$uid \
--gid=$gid
-cut--
If
Re: [Freeipa-users] Export user and host list to a csv or text file
Is the Python API documented anywhere? I've looked around without success. On 05/23/2014 07:54 AM, Martin Kosek wrote: On 05/23/2014 06:42 AM, Sanju A wrote: Dear All, Is there any command to export the user and host list to a csv or text format There is no such command out of the shelf, I would personally just write a short Python script to export the hosts (or anything else) in a format I need. Example for host: ~ #!/usr/bin/python2 from ipalib import api api.bootstrap(context='exporter', debug=False) api.finalize() api.Backend.xmlclient.connect() hosts = api.Command['host_find']()['result'] for host in hosts: print host['fqdn'][0] ~ This will print one host for each new line. Martin ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] LDAP/SSSD/IPA performance
Collecting my various threads together under one big issue and adding this new data point: Our web UI on our slow network is exhibiting some strange behavior as well. When selecting, for example, the Users, it can take up to 5 seconds to fetch 20 out of our 56 entries. When switching to Hosts, it took 4 seconds for the footer to show that there would be 47 pages in total, then after 10 seconds total, the page loaded 20 of 939 entries. When I select a host, the previously-selected host will actually be displayed for upwards of 8-10 seconds (while the spinning cursor spins near the word Logout) until the host actually loads. Is it just me, or does this, plus everything else, start to sound like LDAP is struggling? I ran a test using ldapsearch in authenticated and unauthenticated mode from my workstation and here's what I found, which may tell us nothing: # time ldapsearch -x -H -ldap://zsipa.foo.net base=uid=bretw,cn=users,cn=accounts,dc=foo,dc=net : real0m2.047s user 0m0.000s sys 0m0.001s # time ldapsearch -Y GSSAPI -H ldap://zsipa.foo.net base=uid=bretw,cn=users,cn=accounts,dc=foo,dc=net : real0m2.816s user 0m0.004s sys 0m0.002s When I did this locally on the ipa master: # ssh zsipa.foo.net # time ldapsearch -Y GSSAPI base=uid=bretw,cn=uses,cn=accounts,dc=foo,dc=net : real0m0.847s user 0m0.007s sys 0m0.006s # -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] LDAP/SSSD/IPA performance
More soft/anecdotal: When executing sudo -i or sudo -iu the first time, we can expect a several second delay before the command completes. If we then exit the session and re-execute the command, it will complete almost instantly. So whatever cache is holding this information, if we could increase its duration, that would certainly make our pain less. Is this a settable value? Entering a password into a screensaver is particularly painful. 10+ seconds before the screensaver will exit. We are looking at environmental possibilities, like interfaces and such. This machine is running on a VMware VM, but we've had success deploying IPA on VMs in the past, and our faster network is running VMs as well (with one physical box). Bret On 05/23/2014 08:15 AM, Bret Wortman wrote: Collecting my various threads together under one big issue and adding this new data point: Our web UI on our slow network is exhibiting some strange behavior as well. When selecting, for example, the Users, it can take up to 5 seconds to fetch 20 out of our 56 entries. When switching to Hosts, it took 4 seconds for the footer to show that there would be 47 pages in total, then after 10 seconds total, the page loaded 20 of 939 entries. When I select a host, the previously-selected host will actually be displayed for upwards of 8-10 seconds (while the spinning cursor spins near the word Logout) until the host actually loads. Is it just me, or does this, plus everything else, start to sound like LDAP is struggling? I ran a test using ldapsearch in authenticated and unauthenticated mode from my workstation and here's what I found, which may tell us nothing: # time ldapsearch -x -H -ldap://zsipa.foo.net base=uid=bretw,cn=users,cn=accounts,dc=foo,dc=net : real0m2.047s user 0m0.000s sys 0m0.001s # time ldapsearch -Y GSSAPI -H ldap://zsipa.foo.net base=uid=bretw,cn=users,cn=accounts,dc=foo,dc=net : real0m2.816s user 0m0.004s sys 0m0.002s When I did this locally on the ipa master: # ssh zsipa.foo.net # time ldapsearch -Y GSSAPI base=uid=bretw,cn=uses,cn=accounts,dc=foo,dc=net : real0m0.847s user 0m0.007s sys 0m0.006s # -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] LDAP/SSSD/IPA performance
I assumed. It obviously hasn't helped our sudo situation, but I wouldn't expect it to. I'll let you know how it plays against screensavers and such. On 05/23/2014 10:05 AM, Jakub Hrozek wrote: On Fri, May 23, 2014 at 04:03:44PM +0200, Jakub Hrozek wrote: On Fri, May 23, 2014 at 09:48:00AM -0400, Bret Wortman wrote: More soft/anecdotal: When executing sudo -i or sudo -iu the first time, we can expect a several second delay before the command completes. If we then exit the session and re-execute the command, it will complete almost instantly. So whatever cache is holding this information, if we could increase its duration, that would certainly make our pain less. Is this a settable value? Entering a password into a screensaver is particularly painful. 10+ seconds before the screensaver will exit. We are looking at environmental possibilities, like interfaces and such. This machine is running on a VMware VM, but we've had success deploying IPA on VMs in the past, and our faster network is running VMs as well (with one physical box). Can you try increasing this option: pam_id_timeout (integer) For any PAM request while SSSD is online, the SSSD will attempt to immediately update the cached identity information for the user in order to ensure that authentication takes place with the latest information. A complete PAM conversation may perform multiple PAM requests, such as account management and session opening. This option controls (on a per-client-application basis) how long (in seconds) we can cache the identity information to avoid excessive round-trips to the identity provider. Default: 5 I should also have explicitly said that the option belongs to the [pam] section. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] LDAP/SSSD/IPA performance
All I saw was additional output when I ran the command. On the slower system, there was a one second lag, then a burst of activity, then a one second lag, then completion. I’ll do it again Monday and see what the logs show. On May 23, 2014, at 2:44 PM, Dmitri Pal d...@redhat.com wrote: On 05/23/2014 10:03 AM, Bret Wortman wrote: On 05/23/2014 09:53 AM, Mauricio Tavares wrote: On Fri, May 23, 2014 at 9:48 AM, Bret Wortman bret.wort...@damascusgrp.com wrote: More soft/anecdotal: When executing sudo -i or sudo -iu the first time, we can expect a several second delay before the command completes. If we then exit the session and re-execute the command, it will complete almost instantly. So whatever cache is holding this information, if we could increase its duration, that would certainly make our pain less. Is this a settable value? Entering a password into a screensaver is particularly painful. 10+ seconds before the screensaver will exit. We are looking at environmental possibilities, like interfaces and such. This machine is running on a VMware VM, but we've had success deploying IPA on VMs in the past, and our faster network is running VMs as well (with one physical box). Bret Did running sudo in debugging mode (SUDOERS_DEBUG 2 in ldap.conf) give you any more clues? No. I compared the output on both networks and there's no real difference once I accounted for HBAC on one (which produced 2 entries on the slower network that got filtered down to 1 user match and 1 host match). But the debug output was nearly identical. Did you see any gaps in time in the logs that are different? The flow can be the same but some operations can take longer so there would be hint to us on what to look for. On 05/23/2014 08:15 AM, Bret Wortman wrote: Collecting my various threads together under one big issue and adding this new data point: Our web UI on our slow network is exhibiting some strange behavior as well. When selecting, for example, the Users, it can take up to 5 seconds to fetch 20 out of our 56 entries. When switching to Hosts, it took 4 seconds for the footer to show that there would be 47 pages in total, then after 10 seconds total, the page loaded 20 of 939 entries. When I select a host, the previously-selected host will actually be displayed for upwards of 8-10 seconds (while the spinning cursor spins near the word Logout) until the host actually loads. Is it just me, or does this, plus everything else, start to sound like LDAP is struggling? I ran a test using ldapsearch in authenticated and unauthenticated mode from my workstation and here's what I found, which may tell us nothing: # time ldapsearch -x -H -ldap://zsipa.foo.net base=uid=bretw,cn=users,cn=accounts,dc=foo,dc=net : real0m2.047s user 0m0.000s sys 0m0.001s # time ldapsearch -Y GSSAPI -H ldap://zsipa.foo.net base=uid=bretw,cn=users,cn=accounts,dc=foo,dc=net : real0m2.816s user 0m0.004s sys 0m0.002s When I did this locally on the ipa master: # ssh zsipa.foo.net # time ldapsearch -Y GSSAPI base=uid=bretw,cn=uses,cn=accounts,dc=foo,dc=net : real0m0.847s user 0m0.007s sys 0m0.006s # -- Bret Wortman Mail Attachment.png http://damascusgrp.com/ http://about.me/wortmanbret ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] openldap certs?
Where should my clients be getting the contents of /etc/openldap/certs from? I've got one network where my IPA authentications are blazing fast and one where they're ... not. On the slower one, clients' /etc/openldap/certs directories are either missing or empty; on the faster network, clients have certs in these directories. Is this important, and if so what could be going wrong on my slower network that might cause the certs to not get distributed or created properly? -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] openldap certs?
What we're seeing is slow GDM logins, ssh authentications, and sudo -i responses on this network. On our other, these things are all blazing fast. Here, they're on the order of 5-10 seconds. And it doesn't seem to improve (much) with age or time, except perhaps anecdotally. At best, a second connection might be a second faster, but will revert within an hour or so. On 05/22/2014 09:36 AM, Rob Crittenden wrote: Bret Wortman wrote: Where should my clients be getting the contents of /etc/openldap/certs from? I've got one network where my IPA authentications are blazing fast and one where they're ... not. On the slower one, clients' /etc/openldap/certs directories are either missing or empty; on the faster network, clients have certs in these directories. Is this important, and if so what could be going wrong on my slower network that might cause the certs to not get distributed or created properly? These are not the droids you are looking for... Can you clarify what you mean by IPA authentications? sssd should be handling that, and while a first auth over a slow link might be slow subsequent usage should be quite fast. rob smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] openldap certs?
I found that our slower system was using FQDNs for the list of IPA servers; our faster system was using IPs. I'm switching now, letting Puppet distribute the update and will see if it helps. By enumeration, do you mean are we spelling out our IPA servers? Yes. We only have 3 and they look something like this: [domain/foo.net] cache_credentials = True krb5_store_password_if_offline = True ipa_domain = foo.net id_provider = ipa auth_provider = ipa access_provider = ipa ipa_hostname = rm266ws-a.foo.net chpass_provider = ipa ipa_dyndns_update = True ipa_server = _srv_, 192.168.2.61, 192.168.2.62, 192.168.2.63 ldap_netgroup_search_base = cn=ng,cn=compat,dc=foo,dc=net ldap_tls_cacert = /etc/ipa/ca.crt [sssd] services = nss, pam, ssh config_file_version = 2 domains = foo.net [nss] [pam] [sudo] [autofs] [ssh] [pac] On the other hand, if you meant something else, then I hope the answer's in the file. ;-) On 05/22/2014 10:15 AM, Dmitri Pal wrote: On 05/22/2014 09:43 AM, Bret Wortman wrote: What we're seeing is slow GDM logins, ssh authentications, and sudo -i responses on this network. On our other, these things are all blazing fast. Here, they're on the order of 5-10 seconds. And it doesn't seem to improve (much) with age or time, except perhaps anecdotally. At best, a second connection might be a second faster, but will revert within an hour or so. Have you compared sssd.conf from clients in these two networks? Do you use enumeration? Increasing debug level and looking at the logs will help you to understand what part takes most time. These logs will be helpful for you/us to see if/what the problem is/are. On 05/22/2014 09:36 AM, Rob Crittenden wrote: Bret Wortman wrote: Where should my clients be getting the contents of /etc/openldap/certs from? I've got one network where my IPA authentications are blazing fast and one where they're ... not. On the slower one, clients' /etc/openldap/certs directories are either missing or empty; on the faster network, clients have certs in these directories. Is this important, and if so what could be going wrong on my slower network that might cause the certs to not get distributed or created properly? These are not the droids you are looking for... Can you clarify what you mean by IPA authentications? sssd should be handling that, and while a first auth over a slow link might be slow subsequent usage should be quite fast. rob ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] New replica won't accept replication
Go figure. I rebuilt it (again) cleanly, and after starting replication again, while I was madly trying to change the debug level on the new replica...it completed replication this time. Heisenbugs. Gotta love them. (I think this one was in my network somewhere, not your code -- I just couldn't observe it enough and someone must've changed something while I wasn't looking). Bret On 05/21/2014 10:19 PM, Rob Crittenden wrote: Bret Wortman wrote: It takes about 2 minutes. How would you like me to turn debugging on? http://directory.fedoraproject.org/wiki/FAQ#Troubleshooting I'm not sure if you should enable this on both sides of the agreement or not. If you have the ability and don't mind potentially slowing down the working master it might be useful to the 389-ds guys. rob Bret Wortman http://bretwortman.com/ http://twitter.com/BretWortman On May 21, 2014, at 4:26 PM, Rob Crittenden rcrit...@redhat.com wrote: Bret Wortman wrote: On the new replica (asipa) I see in the access log almost 5000 entries like this: [21/May/2014:10:30:58 -0400] conn=4 op=4923 EXT oid=2.16.840.113730.3.5.6 name=Netscape Replication Total update Entry [21/May/2014:10:30:58 -0400] conn=4 op=4923 RESULT err=0 tag=120 nentries=0 etime=0 And these just repeat, increasing the op value until they terminate with this one. The rest of it just looks like informational messages. How long does this take? Is there time to enable replication debugging? That may provide more output. Over on zsipa (the CA master), errors contains: [21/May/2014:14:31:06 +] NSMMReplciationPlugin - Schema agmt=cn=meToasipa.foo.net (asipa:389) must not be overwritten(set replication log for additional info) [21/May/2014:14:31:06 +] NSMMReplicationPlugin - agmt=cn=meToasipa.foo.net (asipa:389) Warning: unable to replicate schema: rc=1 I don't think this is related. I'd run ipa-replica-manage list -v `hostname` and ipa-csreplica-manage list -v `hostname` on the master you generated the replica install file on to see what agreements it has or thinks it has. rob These two lines repeat at intervals for a while. Nothing else leapt out at me. On 05/21/2014 11:04 AM, Rob Crittenden wrote: Bret Wortman wrote: This occurs on our first attempt to join as a replica. I've erased this box and rebaselined it but the same thing happens. No network ports being blocked that we know of, and another replica I created at the same time installed its replica file without issue. asipa is the new replica, zsipa is the ca and original master on which the replica file was created. [24/34]: setting up initial replication Starting replication, please wait until this has completed Update in progress, 130 seconds elapsed Update in progress yet not in progress [ipamaster.foo.net] reports: Update failed! Status: [10 Total update abortedLDAP error: Referral] Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Failed to start replication # /var/log/ipareplica-install.log contains this: 2014-05-21T145:28:56Z DEBUG retrieving schema for SchemaCache url=ldaps://asipa.fopo.net:636 conn=ldap.ldapobject.SimpleLDAPObject instance at 0x4faf170 2014-05-21T14:31:08Z DEBUG File /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py, line 638, in run_script return_value = main_function() File /usr/sbin/ipa-replica-install, line 663, in main ds = install_replica_ds(config) File /usr/sbin/ipa-replica-install, line 188, in install_replica_ds ca_file=config.dir + /ca.crt, File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 360 in create_replica self.start_creation(runtime=60) File /usr/lib/python2.7/site-packages/ipaserver/install/service.py, line 364, in start_creation method() File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 373, in __setup_replica r_bindpw=self.dm_password() File /usr/lib/python2.7/site-packages/ipaserver/install/replication.py, line 961, in setup_replication raise RuntimeError(Failed to start replication) 2014-0521T14:31:08Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: Failed to start replication Any guidance on where to start looking? Check the 389-ds access and error logs on both masters. rob smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] openldap certs?
It doesn't seem to have helped -- we're still pretty slow even with IP addresses in sssd.conf. On 05/22/2014 11:07 AM, Dmitri Pal wrote: On 05/22/2014 10:36 AM, Bret Wortman wrote: I found that our slower system was using FQDNs for the list of IPA servers; our faster system was using IPs. I'm switching now, letting Puppet distribute the update and will see if it helps. That means you have problems with DNS that are worth looking into. By enumeration, do you mean are we spelling out our IPA servers? Yes. We only have 3 and they look something like this: No. I mean the ability of sssd to download everything when enumerate = true This causes a lot of traffic and overhead and a usual reason for low performance. We were unfortunate to include this setting into one of the early sssd.conf examples and people have been copying it around ever since though we strongly recommend against enabling it. [domain/foo.net] cache_credentials = True krb5_store_password_if_offline = True ipa_domain = foo.net id_provider = ipa auth_provider = ipa access_provider = ipa ipa_hostname = rm266ws-a.foo.net chpass_provider = ipa ipa_dyndns_update = True ipa_server = _srv_, 192.168.2.61, 192.168.2.62, 192.168.2.63 ldap_netgroup_search_base = cn=ng,cn=compat,dc=foo,dc=net ldap_tls_cacert = /etc/ipa/ca.crt [sssd] services = nss, pam, ssh config_file_version = 2 domains = foo.net [nss] [pam] [sudo] [autofs] [ssh] [pac] On the other hand, if you meant something else, then I hope the answer's in the file. ;-) On 05/22/2014 10:15 AM, Dmitri Pal wrote: On 05/22/2014 09:43 AM, Bret Wortman wrote: What we're seeing is slow GDM logins, ssh authentications, and sudo -i responses on this network. On our other, these things are all blazing fast. Here, they're on the order of 5-10 seconds. And it doesn't seem to improve (much) with age or time, except perhaps anecdotally. At best, a second connection might be a second faster, but will revert within an hour or so. Have you compared sssd.conf from clients in these two networks? Do you use enumeration? Increasing debug level and looking at the logs will help you to understand what part takes most time. These logs will be helpful for you/us to see if/what the problem is/are. On 05/22/2014 09:36 AM, Rob Crittenden wrote: Bret Wortman wrote: Where should my clients be getting the contents of /etc/openldap/certs from? I've got one network where my IPA authentications are blazing fast and one where they're ... not. On the slower one, clients' /etc/openldap/certs directories are either missing or empty; on the faster network, clients have certs in these directories. Is this important, and if so what could be going wrong on my slower network that might cause the certs to not get distributed or created properly? These are not the droids you are looking for... Can you clarify what you mean by IPA authentications? sssd should be handling that, and while a first auth over a slow link might be slow subsequent usage should be quite fast. rob ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Why would /etc/passwd get skipped?
If this line is in /etc/nsswitch.conf: passwd: files sss Why would the user account from IPA get used when an identical one exists in /etc/passwd? We can tell because of some additional groups granted when authentication comes from IPA. If I shut down sssd, then login proceeds through /etc/passwd as expected, but as soon as I restart sssd, this behavior starts again. It's almost as if nsswitch.conf is being ignored or read right-to-left. Just another oddity I uncovered on one system as I was troubleshooting a particularly long ssh localhost and trying to rule things out. -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Why would /etc/passwd get skipped?
A. Then it's probably not the source of my performance problem. I know when I shut down SSSD, that user's ssh times speed up incredibly. Bret On 05/22/2014 01:06 PM, Simo Sorce wrote: On Thu, 2014-05-22 at 12:47 -0400, Bret Wortman wrote: If this line is in /etc/nsswitch.conf: passwd: files sss Why would the user account from IPA get used when an identical one exists in /etc/passwd? We can tell because of some additional groups granted when authentication comes from IPA. If I shut down sssd, then login proceeds through /etc/passwd as expected, but as soon as I restart sssd, this behavior starts again. It's almost as if nsswitch.conf is being ignored or read right-to-left. Just another oddity I uncovered on one system as I was troubleshooting a particularly long ssh localhost and trying to rule things out. The initgroups call (done at authentication to find what groups a user is member of) by default traverses all databases, so if the same username is found in multiple databases the groups are added as well. There is actually a way to change this behavior, although it usually causes more issue than it resolves. You could try with: initgroups: files sss Simo. smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Why would /etc/passwd get skipped?
Yep, that initgroups change had the same effect as shutting down sssd, but without inconveniencing all the IPA-only users. The problem in this particular case was made worse by a lot of network latency, but even on network segments local to the ipa masters, it's taking seconds to authenticate. This will help out the local accounts, at least. Now to keep working on those that aren't local. Thanks for that tip, Simo! On 05/22/2014 01:15 PM, Simo Sorce wrote: On Thu, 2014-05-22 at 13:12 -0400, Bret Wortman wrote: A. Then it's probably not the source of my performance problem. I know when I shut down SSSD, that user's ssh times speed up incredibly. This makes me think it *is* initgroups, as it normally will hit sssd even for non-sssd owned users. But the issue here clearly is that sssd is slow for you, bad network ? Simo. Bret On 05/22/2014 01:06 PM, Simo Sorce wrote: On Thu, 2014-05-22 at 12:47 -0400, Bret Wortman wrote: If this line is in /etc/nsswitch.conf: passwd: files sss Why would the user account from IPA get used when an identical one exists in /etc/passwd? We can tell because of some additional groups granted when authentication comes from IPA. If I shut down sssd, then login proceeds through /etc/passwd as expected, but as soon as I restart sssd, this behavior starts again. It's almost as if nsswitch.conf is being ignored or read right-to-left. Just another oddity I uncovered on one system as I was troubleshooting a particularly long ssh localhost and trying to rule things out. The initgroups call (done at authentication to find what groups a user is member of) by default traverses all databases, so if the same username is found in multiple databases the groups are added as well. There is actually a way to change this behavior, although it usually causes more issue than it resolves. You could try with: initgroups: files sss Simo. smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] New replica won't accept replication
This occurs on our first attempt to join as a replica. I've erased this box and rebaselined it but the same thing happens. No network ports being blocked that we know of, and another replica I created at the same time installed its replica file without issue. asipa is the new replica, zsipa is the ca and original master on which the replica file was created. [24/34]: setting up initial replication Starting replication, please wait until this has completed Update in progress, 130 seconds elapsed Update in progress yet not in progress [ipamaster.foo.net] reports: Update failed! Status: [10 Total update abortedLDAP error: Referral] Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Failed to start replication # /var/log/ipareplica-install.log contains this: 2014-05-21T145:28:56Z DEBUG retrieving schema for SchemaCache url=ldaps://asipa.fopo.net:636 conn=ldap.ldapobject.SimpleLDAPObject instance at 0x4faf170 2014-05-21T14:31:08Z DEBUG File /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py, line 638, in run_script return_value = main_function() File /usr/sbin/ipa-replica-install, line 663, in main ds = install_replica_ds(config) File /usr/sbin/ipa-replica-install, line 188, in install_replica_ds ca_file=config.dir + /ca.crt, File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 360 in create_replica self.start_creation(runtime=60) File /usr/lib/python2.7/site-packages/ipaserver/install/service.py, line 364, in start_creation method() File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 373, in __setup_replica r_bindpw=self.dm_password() File /usr/lib/python2.7/site-packages/ipaserver/install/replication.py, line 961, in setup_replication raise RuntimeError(Failed to start replication) 2014-0521T14:31:08Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: Failed to start replication Any guidance on where to start looking? -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] New replica won't accept replication
On the new replica (asipa) I see in the access log almost 5000 entries like this: [21/May/2014:10:30:58 -0400] conn=4 op=4923 EXT oid=2.16.840.113730.3.5.6 name=Netscape Replication Total update Entry [21/May/2014:10:30:58 -0400] conn=4 op=4923 RESULT err=0 tag=120 nentries=0 etime=0 And these just repeat, increasing the op value until they terminate with this one. The rest of it just looks like informational messages. Over on zsipa (the CA master), errors contains: [21/May/2014:14:31:06 +] NSMMReplciationPlugin - Schema agmt=cn=meToasipa.foo.net (asipa:389) must not be overwritten(set replication log for additional info) [21/May/2014:14:31:06 +] NSMMReplicationPlugin - agmt=cn=meToasipa.foo.net (asipa:389) Warning: unable to replicate schema: rc=1 These two lines repeat at intervals for a while. Nothing else leapt out at me. On 05/21/2014 11:04 AM, Rob Crittenden wrote: Bret Wortman wrote: This occurs on our first attempt to join as a replica. I've erased this box and rebaselined it but the same thing happens. No network ports being blocked that we know of, and another replica I created at the same time installed its replica file without issue. asipa is the new replica, zsipa is the ca and original master on which the replica file was created. [24/34]: setting up initial replication Starting replication, please wait until this has completed Update in progress, 130 seconds elapsed Update in progress yet not in progress [ipamaster.foo.net] reports: Update failed! Status: [10 Total update abortedLDAP error: Referral] Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Failed to start replication # /var/log/ipareplica-install.log contains this: 2014-05-21T145:28:56Z DEBUG retrieving schema for SchemaCache url=ldaps://asipa.fopo.net:636 conn=ldap.ldapobject.SimpleLDAPObject instance at 0x4faf170 2014-05-21T14:31:08Z DEBUG File /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py, line 638, in run_script return_value = main_function() File /usr/sbin/ipa-replica-install, line 663, in main ds = install_replica_ds(config) File /usr/sbin/ipa-replica-install, line 188, in install_replica_ds ca_file=config.dir + /ca.crt, File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 360 in create_replica self.start_creation(runtime=60) File /usr/lib/python2.7/site-packages/ipaserver/install/service.py, line 364, in start_creation method() File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 373, in __setup_replica r_bindpw=self.dm_password() File /usr/lib/python2.7/site-packages/ipaserver/install/replication.py, line 961, in setup_replication raise RuntimeError(Failed to start replication) 2014-0521T14:31:08Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: Failed to start replication Any guidance on where to start looking? Check the 389-ds access and error logs on both masters. rob smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] New replica won't accept replication
...but it did at least look like they were talking, right? Some level of replication was happening: (before the Netscape Replication Total update Entry began running away with the logfile): [21/May/2014:10:28:52 -0400] conn=2 op=2 RESULT err=0 tag=101 nentries=1 etime=0 [21/May/2014:10:28:53 -0400] conn=2 op=3 MOD dn=cn=IPA Version Replication,cn=Plugins,cn=config [21/May/2014:10:28:53 -0400] conn=2 op=3 RESULT err=0 tag=103 nentries=0 etime=0 [21/May/2014:10:28:53 -0400] conn=2 op=4 UNBIND On 05/21/2014 11:40 AM, Bret Wortman wrote: On the new replica (asipa) I see in the access log almost 5000 entries like this: [21/May/2014:10:30:58 -0400] conn=4 op=4923 EXT oid=2.16.840.113730.3.5.6 name=Netscape Replication Total update Entry [21/May/2014:10:30:58 -0400] conn=4 op=4923 RESULT err=0 tag=120 nentries=0 etime=0 And these just repeat, increasing the op value until they terminate with this one. The rest of it just looks like informational messages. Over on zsipa (the CA master), errors contains: [21/May/2014:14:31:06 +] NSMMReplciationPlugin - Schema agmt=cn=meToasipa.foo.net (asipa:389) must not be overwritten(set replication log for additional info) [21/May/2014:14:31:06 +] NSMMReplicationPlugin - agmt=cn=meToasipa.foo.net (asipa:389) Warning: unable to replicate schema: rc=1 These two lines repeat at intervals for a while. Nothing else leapt out at me. On 05/21/2014 11:04 AM, Rob Crittenden wrote: Bret Wortman wrote: This occurs on our first attempt to join as a replica. I've erased this box and rebaselined it but the same thing happens. No network ports being blocked that we know of, and another replica I created at the same time installed its replica file without issue. asipa is the new replica, zsipa is the ca and original master on which the replica file was created. [24/34]: setting up initial replication Starting replication, please wait until this has completed Update in progress, 130 seconds elapsed Update in progress yet not in progress [ipamaster.foo.net] reports: Update failed! Status: [10 Total update abortedLDAP error: Referral] Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Failed to start replication # /var/log/ipareplica-install.log contains this: 2014-05-21T145:28:56Z DEBUG retrieving schema for SchemaCache url=ldaps://asipa.fopo.net:636 conn=ldap.ldapobject.SimpleLDAPObject instance at 0x4faf170 2014-05-21T14:31:08Z DEBUG File /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py, line 638, in run_script return_value = main_function() File /usr/sbin/ipa-replica-install, line 663, in main ds = install_replica_ds(config) File /usr/sbin/ipa-replica-install, line 188, in install_replica_ds ca_file=config.dir + /ca.crt, File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 360 in create_replica self.start_creation(runtime=60) File /usr/lib/python2.7/site-packages/ipaserver/install/service.py, line 364, in start_creation method() File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 373, in __setup_replica r_bindpw=self.dm_password() File /usr/lib/python2.7/site-packages/ipaserver/install/replication.py, line 961, in setup_replication raise RuntimeError(Failed to start replication) 2014-0521T14:31:08Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: Failed to start replication Any guidance on where to start looking? Check the 389-ds access and error logs on both masters. rob ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] New replica won't accept replication
It takes about 2 minutes. How would you like me to turn debugging on? Bret Wortman http://bretwortman.com/ http://twitter.com/BretWortman On May 21, 2014, at 4:26 PM, Rob Crittenden rcrit...@redhat.com wrote: Bret Wortman wrote: On the new replica (asipa) I see in the access log almost 5000 entries like this: [21/May/2014:10:30:58 -0400] conn=4 op=4923 EXT oid=2.16.840.113730.3.5.6 name=Netscape Replication Total update Entry [21/May/2014:10:30:58 -0400] conn=4 op=4923 RESULT err=0 tag=120 nentries=0 etime=0 And these just repeat, increasing the op value until they terminate with this one. The rest of it just looks like informational messages. How long does this take? Is there time to enable replication debugging? That may provide more output. Over on zsipa (the CA master), errors contains: [21/May/2014:14:31:06 +] NSMMReplciationPlugin - Schema agmt=cn=meToasipa.foo.net (asipa:389) must not be overwritten(set replication log for additional info) [21/May/2014:14:31:06 +] NSMMReplicationPlugin - agmt=cn=meToasipa.foo.net (asipa:389) Warning: unable to replicate schema: rc=1 I don't think this is related. I'd run ipa-replica-manage list -v `hostname` and ipa-csreplica-manage list -v `hostname` on the master you generated the replica install file on to see what agreements it has or thinks it has. rob These two lines repeat at intervals for a while. Nothing else leapt out at me. On 05/21/2014 11:04 AM, Rob Crittenden wrote: Bret Wortman wrote: This occurs on our first attempt to join as a replica. I've erased this box and rebaselined it but the same thing happens. No network ports being blocked that we know of, and another replica I created at the same time installed its replica file without issue. asipa is the new replica, zsipa is the ca and original master on which the replica file was created. [24/34]: setting up initial replication Starting replication, please wait until this has completed Update in progress, 130 seconds elapsed Update in progress yet not in progress [ipamaster.foo.net] reports: Update failed! Status: [10 Total update abortedLDAP error: Referral] Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Failed to start replication # /var/log/ipareplica-install.log contains this: 2014-05-21T145:28:56Z DEBUG retrieving schema for SchemaCache url=ldaps://asipa.fopo.net:636 conn=ldap.ldapobject.SimpleLDAPObject instance at 0x4faf170 2014-05-21T14:31:08Z DEBUG File /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py, line 638, in run_script return_value = main_function() File /usr/sbin/ipa-replica-install, line 663, in main ds = install_replica_ds(config) File /usr/sbin/ipa-replica-install, line 188, in install_replica_ds ca_file=config.dir + /ca.crt, File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 360 in create_replica self.start_creation(runtime=60) File /usr/lib/python2.7/site-packages/ipaserver/install/service.py, line 364, in start_creation method() File /usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py, line 373, in __setup_replica r_bindpw=self.dm_password() File /usr/lib/python2.7/site-packages/ipaserver/install/replication.py, line 961, in setup_replication raise RuntimeError(Failed to start replication) 2014-0521T14:31:08Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: Failed to start replication Any guidance on where to start looking? Check the 389-ds access and error logs on both masters. rob smime.p7s Description: S/MIME cryptographic signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] IPA down hard. Kerberos?
Happy Monday to me -- I came in this morning to find all 3 of my IPA replicas are down. When I tried to start one of them, I got this: [root@ipa1 ~]# ipactl start Existing service file detected! Assuming stale, cleaning and proceeding Starting Directory Service Starting krb5kdc Service Job for krb5kdc.service failed. See 'systemctl status krb5kdc.service' and 'journalctl -xn' for details. Failed to start krb5kdc Service Shutting down Aborting ipactl [root@ipa1 ~]# systemctl status krb5kdc.service krb5kdc.service - Kerberos 5 KDC Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service; disabled) Active: failed (Result: exit-code) since Mon 2014-05-19 06:46:24 EDT; 51s ago Process: 1835 ExecStart=/usr/sbin/krb5kdc -P /var/run/krb5kdc.pid $KRB5KDC_ARGS (code=exited, status=1/FAILURE) May 19 06:46:24 ipa1.foo.net systemd[1]: krb5kdc.service: control process exited, code=exited status=1 May 19 06:46:24 ipa1.foo.net systemd[1]: Failed to start Kerberos 5 KDC. May 19 06:46:24 ipa1.foo.net systemd[1]: Unit krb5kdc.service entered failed state. May 19 06:46:24 ipa1.foo.net systemd[1]: Stopped Kerberos 5 KDC. [root@ipa1 ~]# journalctl -xn -- Logs begin at Tue 2014-05-13 09:50:44 EDT, end at Mon 2014-05-19 06:47:03 EDT. -- May 19 06:46:42 ipa1.foo.net ntpd_intres[526]: host name not found: 2.fedora.pool.ntp.org May 19 06:46:58 ipa1.foo.net sshd[1855]: error: AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys returned status 1 May 19 06:47:00 ipa1.foo.net sshd[1855]: Accepted password for root from 192.168.2.13 port 42299 ssh2 May 19 06:47:00 ipa1.foo.net systemd[1]: Starting Session 5 of user root. -- Subject: Unit session-5.scope has begun with start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit session-5.scope has begun starting up. May 19 06:47:00 ipa1.foo.net systemd-logind[495]: New session 5 of user root. -- Subject: A new session 5 has been created for user root -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- Documentation: http://www.freedesktop.org/wiki/Software/systemd/multiseat -- -- A new session with the ID 5 has been created for the user root. -- -- The leading process of the session is 1855. May 19 06:47:00 ipa1.foo.net systemd[1]: Started Session 5 of user root. -- Subject: Unit session-5.scope has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit session-5.scope has finished starting up. -- -- The start-up result is done. May 19 06:47:00 ipa1.foo.net sshd[1855]: pam_unix(sshd:session): session opened for user root by (uid=0) May 19 06:47:03 ipa1.foo.net systemd[1]: Stopped 389 Directory Server WEDGEOFLI-ME.. -- Subject: Unit dirsrv@WEDGEOFLI-ME.service has finished shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit dirsrv@WEDGEOFLI-ME.service has finished shutting down. May 19 06:47:03 ipa1.foo.net systemd[1]: Stopping 389 Directory Server. -- Subject: Unit dirsrv.target has begun shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit dirsrv.target has begun shutting down. May 19 06:47:03 ipa1.foo.net systemd[1]: Stopped target 389 Directory Server. -- Subject: Unit dirsrv.target has finished shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit dirsrv.target has finished shutting down. [root@ipa1 ~]# Any thoughts on where to look next? There's nothing at all logged in /var/log/krb5kdc.log when I try to start it up, and there are so many pieces to this that I'm not sure where to focus my efforts. Thanks! -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Best practices for core servers
I can already see from this that our key problem may have been that we had one server functioning as the hub and every other remote replica had just one agreement, but those agreements were all with the hub. So that hub had ten agreements. Badness. We'll give this some good attention as we move forward. Thanks for the pointer, Martin. Bret On 04/30/2014 03:15 AM, Martin Kosek wrote: On 04/28/2014 01:03 PM, Bret Wortman wrote: We are planning to reconfigure our core Freeipa servers, basically building a replacement infrastructure and migrating to it. What we're planning right now is a core of three Freeipa servers each of which has a CA, with as much distribution of replication as we can manage. I imagine that means one of them replicates to the other two but am open to other ideas. You can configure them to replica to each other. For remote locations, we're planning to stand up caching-only DNS servers, as authenticating back to the main IPA servers works extremely well; it's just DNS that needs a little help. Any thoughts before I start setting these servers (VMs, most likely) up? You may want to read our upstream Deployment Recommendations article, it may save you some bad decisions from the start: http://www.freeipa.org/page/Deployment_Recommendations If we see that we missed anything in this article, it would be great to enhance it. Martin smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Switching a client from one set of IPA servers to another
I'd like to test migrating our clients from the old IPA infrastructure
to our newer F20-based servers but am having trouble with our first
clients. Unenrolling them from the old IPA servers went fine, but when I
try to enroll them with the newer ones, the logs report:
# ipa-client-install -U --server zsipa.foo.net --domain foo.net
--password obscured --mkhomdir --enable-dns-updates
LDAP Error: Connect error: TLS error -8172:Peer's certificate issuer has
been marked as not trusted by the user.
LDAP Error: Connect error: TLS error -8172:Peer's certificate issuer has
been marked as not trusted by the user.
Failed to verify that zsipa.foo.net is an IPA Server.
This may mean that the remote server is not up or is not reachable due
to network or firewall settings.
:
:
Installation failed. Rolling back changes.
IPA client is not configured on this system.
# ps aux | grep firewalld| grep -v grep
# getenforce
Disabled
# cat /var/log/ipaclient-install.log
:
:
DEBUG [LDAP server check]
DEBUG Verifying that zsipa.foo.net (realm foo.net) is an IPA server
DEBUG Init LDAP connection with: ldap://zsipa.foo.net:389
ERROR LDAP Error: Connect error: TLS error -8173:Peer's certificate
issuer has been marked as not trusted by the user.
DEBUG Discovery result: UNKNOWN_ERROR; server=None, domain=foo.net,
kdc=zsipa.foo.net, basedn=None
DEBUG Validated servers:
DEBUG will use discovered domain: foo.net
DEBUG IPA Server not found
DEBUG [IPA Discovery] Starting IPA discovery with domain=foo.net,
servers=['zsipa.foo.net'], hostname=jsutil.foo.net
DEBUG Server and domain forced
DEBUG [Kerberos realm search]
DEBUG Search DNS for TXT record of _kerberos.foo.net
DEBUG DNS record found:
DNSResult::name:_kerberos.foo.net.,type:16,class:1,rdata={data:FOO.NET}
DEBUG Search DNS for SRV record of
_kerberos._udp.foo.net.,type:33,class:1,rdata={priority:0,port:88,weight:100,server:zsipa.foo.net.}
DEBUG [LDAP server check]
DEBUG Verifying that zsipa.foo.net (realm FOO.NET)is an IPA server
DEBUG Init LDAP connection with: ldap://zsipa.foo.net:389
ERROR LDAP Error: Connect error: TLS error -8172:Peer's certificate
issuer has been marked as not trusted by the user.
DEBUG Discovery result: UNKNOWN_ERROR; server=None, domain=foo.net,
kdc=zsipa.foo.net, basedn=None
DEBUG Validated servers:
ERROR Failed to verify that zsipa.foo.net is an IPA Server.
ERROR This may mean that the remote server is not up or is not reachable
due to network or firewall settings.
INFO Please make sure the following ports are opened in the firewall
settings:
TCP: 80, 88, 389
UDP: 88 (at least one of TCP/UDP ports 88 has to be open)
Also note that following ports are necessary for ipa-client working
properly after enrollment:
TCP: 464
UDP: 464, 123 (if NTP enabled)
DEBUG (zspia.foo.net: Provided as option)
ERROR Installation failed. Rolling back changes.
ERROR IPA client is not configured on this system.
I removed the timestamps for readability.
It seems to me that something from the old version is hanging around and
getting in the way, or that something in the setup of the new server
isn't quite complete -- which seems more likely, and where should I be
looking for the actual cause? Is this a problem with a certificate or
with the server not being discoverable?
--
*Bret Wortman*
http://damascusgrp.com/
http://about.me/wortmanbret
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Switching a client from one set of IPA servers to another
Crap. Thought I caught this before I sent it.
# rm -f /etc/ipa/ca.crt
On 04/29/2014 01:22 PM, Bret Wortman wrote:
I'd like to test migrating our clients from the old IPA infrastructure
to our newer F20-based servers but am having trouble with our first
clients. Unenrolling them from the old IPA servers went fine, but when
I try to enroll them with the newer ones, the logs report:
# ipa-client-install -U --server zsipa.foo.net --domain foo.net
--password obscured --mkhomdir --enable-dns-updates
LDAP Error: Connect error: TLS error -8172:Peer's certificate issuer
has been marked as not trusted by the user.
LDAP Error: Connect error: TLS error -8172:Peer's certificate issuer
has been marked as not trusted by the user.
Failed to verify that zsipa.foo.net is an IPA Server.
This may mean that the remote server is not up or is not reachable due
to network or firewall settings.
:
:
Installation failed. Rolling back changes.
IPA client is not configured on this system.
# ps aux | grep firewalld| grep -v grep
# getenforce
Disabled
# cat /var/log/ipaclient-install.log
:
:
DEBUG [LDAP server check]
DEBUG Verifying that zsipa.foo.net (realm foo.net) is an IPA server
DEBUG Init LDAP connection with: ldap://zsipa.foo.net:389
ERROR LDAP Error: Connect error: TLS error -8173:Peer's certificate
issuer has been marked as not trusted by the user.
DEBUG Discovery result: UNKNOWN_ERROR; server=None, domain=foo.net,
kdc=zsipa.foo.net, basedn=None
DEBUG Validated servers:
DEBUG will use discovered domain: foo.net
DEBUG IPA Server not found
DEBUG [IPA Discovery] Starting IPA discovery with domain=foo.net,
servers=['zsipa.foo.net'], hostname=jsutil.foo.net
DEBUG Server and domain forced
DEBUG [Kerberos realm search]
DEBUG Search DNS for TXT record of _kerberos.foo.net
DEBUG DNS record found:
DNSResult::name:_kerberos.foo.net.,type:16,class:1,rdata={data:FOO.NET}
DEBUG Search DNS for SRV record of
_kerberos._udp.foo.net.,type:33,class:1,rdata={priority:0,port:88,weight:100,server:zsipa.foo.net.}
DEBUG [LDAP server check]
DEBUG Verifying that zsipa.foo.net (realm FOO.NET)is an IPA server
DEBUG Init LDAP connection with: ldap://zsipa.foo.net:389
ERROR LDAP Error: Connect error: TLS error -8172:Peer's certificate
issuer has been marked as not trusted by the user.
DEBUG Discovery result: UNKNOWN_ERROR; server=None, domain=foo.net,
kdc=zsipa.foo.net, basedn=None
DEBUG Validated servers:
ERROR Failed to verify that zsipa.foo.net is an IPA Server.
ERROR This may mean that the remote server is not up or is not
reachable due to network or firewall settings.
INFO Please make sure the following ports are opened in the firewall
settings:
TCP: 80, 88, 389
UDP: 88 (at least one of TCP/UDP ports 88 has to be open)
Also note that following ports are necessary for ipa-client working
properly after enrollment:
TCP: 464
UDP: 464, 123 (if NTP enabled)
DEBUG (zspia.foo.net: Provided as option)
ERROR Installation failed. Rolling back changes.
ERROR IPA client is not configured on this system.
I removed the timestamps for readability.
It seems to me that something from the old version is hanging around
and getting in the way, or that something in the setup of the new
server isn't quite complete -- which seems more likely, and where
should I be looking for the actual cause? Is this a problem with a
certificate or with the server not being discoverable?
--
*Bret Wortman*
http://damascusgrp.com/
http://about.me/wortmanbret
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Best practices for core servers
We are planning to reconfigure our core Freeipa servers, basically building a replacement infrastructure and migrating to it. What we're planning right now is a core of three Freeipa servers each of which has a CA, with as much distribution of replication as we can manage. I imagine that means one of them replicates to the other two but am open to other ideas. For remote locations, we're planning to stand up caching-only DNS servers, as authenticating back to the main IPA servers works extremely well; it's just DNS that needs a little help. Any thoughts before I start setting these servers (VMs, most likely) up? -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Error creating new freeipa-server
I'm trying to stand up a new ipa server on a clean box, and I keep
getting this error so _something_ is amiss but I'm not sure what:
:
Configuring certificate server (pki-tomcatd): Estimated time 3 minutes
30 seconds
[1/22]: creating certificate server user
[2/22]: configuring certificate server instance
ipa: CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit status 1
Configuration of CA failed
#
In the /var/log/ipaserver-install.log, I see this:
:
:
Installing CA into /var/lib/pki/pki-tomcat.
Installation failed.
2014-04-28T11:43:46Z DEBUG stderr=pkispawn : ERROR PKI
subsystem 'CA' for instance 'pki-tomcat' already exists!
2014-04-28T11:432:46Z CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit status 1
2014-04-28T11:43:46Z DEBUG File
/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py,
line 622, in run_script
return_value = main_function()
File /usr/sbin/ipa-server-install, line 1074, in main
dm_password, subject_base=options.subject)
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
478, in configure_instance
self.start_creation(runtime=210)
File /usr/lib/python2.7/site-packages/ipaserver/isntall/service.py,
line 364, in start_creation
method()
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
604, in __spawn_instance
raise RUntimeError('Configuration of CA failed')
:
:
So it looks like somehow this has gotten configured already. Possibly
Puppet copied over something it shouldn't have. What do I need to remove
to make this step work without removing so much that I render something
inoperable?
--
*Bret Wortman*
http://damascusgrp.com/
http://about.me/wortmanbret
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
Not to be thick, but what's the best way to check the DS instance for a
pki entry?
On 04/28/2014 07:57 AM, Dmitri Pal wrote:
On 04/28/2014 07:52 AM, Bret Wortman wrote:
I'm trying to stand up a new ipa server on a clean box, and I keep
getting this error so _something_ is amiss but I'm not sure what:
:
Configuring certificate server (pki-tomcatd): Estimated time 3
minutes 30 seconds
[1/22]: creating certificate server user
[2/22]: configuring certificate server instance
ipa: CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit
status 1
Configuration of CA failed
#
In the /var/log/ipaserver-install.log, I see this:
:
:
Installing CA into /var/lib/pki/pki-tomcat.
Installation failed.
2014-04-28T11:43:46Z DEBUG stderr=pkispawn : ERROR
PKI subsystem 'CA' for instance 'pki-tomcat' already exists!
2014-04-28T11:432:46Z CRITICAL failed to configure ca instance
Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned
non-zero exit status 1
2014-04-28T11:43:46Z DEBUG File
/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py,
line 622, in run_script
return_value = main_function()
File /usr/sbin/ipa-server-install, line 1074, in main
dm_password, subject_base=options.subject)
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py,
line 478, in configure_instance
self.start_creation(runtime=210)
File
/usr/lib/python2.7/site-packages/ipaserver/isntall/service.py, line
364, in start_creation
method()
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py,
line 604, in __spawn_instance
raise RUntimeError('Configuration of CA failed')
:
:
So it looks like somehow this has gotten configured already. Possibly
Puppet copied over something it shouldn't have. What do I need to
remove to make this step work without removing so much that I render
something inoperable?
Run uninstall several times. Each time uninstall might clean next
portion and untangle things so trying to do it several times pays off.
Then check if there is a DS instance for PKI. If there is remove it
and try again.
--
*Bret Wortman*
http://damascusgrp.com/
http://about.me/wortmanbret
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
--
Thank you,
Dmitri Pal
Sr. Engineering Manager IdM portfolio
Red Hat, Inc.
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
Great. I'll try that next.
Bret Wortman
http://bretwortman.com/
http://twitter.com/BretWortman
On Apr 28, 2014, at 8:33 AM, Petr Viktorin pvikt...@redhat.com wrote:
On 04/28/2014 01:52 PM, Bret Wortman wrote:
I'm trying to stand up a new ipa server on a clean box, and I keep
getting this error so _something_ is amiss but I'm not sure what:
:
Configuring certificate server (pki-tomcatd): Estimated time 3 minutes
30 seconds
[1/22]: creating certificate server user
[2/22]: configuring certificate server instance
ipa: CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit status 1
Configuration of CA failed
#
In the /var/log/ipaserver-install.log, I see this:
:
:
Installing CA into /var/lib/pki/pki-tomcat.
Installation failed.
2014-04-28T11:43:46Z DEBUG stderr=pkispawn : ERROR PKI
subsystem 'CA' for instance 'pki-tomcat' already exists!
2014-04-28T11:432:46Z CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit status 1
2014-04-28T11:43:46Z DEBUG File
/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py,
line 622, in run_script
return_value = main_function()
File /usr/sbin/ipa-server-install, line 1074, in main
dm_password, subject_base=options.subject)
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
478, in configure_instance
self.start_creation(runtime=210)
File /usr/lib/python2.7/site-packages/ipaserver/isntall/service.py,
line 364, in start_creation
method()
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
604, in __spawn_instance
raise RUntimeError('Configuration of CA failed')
:
:
So it looks like somehow this has gotten configured already. Possibly
Puppet copied over something it shouldn't have. What do I need to remove
to make this step work without removing so much that I render something
inoperable?
According to the error you're getting, there is a CA instance already
installed.
After uninstalling IPA, destroy it with:
pkidestroy -s CA -i pki-tomcat
--
Petr³
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
smime.p7s
Description: S/MIME cryptographic signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
I thought that might be it and didn't see anything but will look again.
Bret Wortman
http://bretwortman.com/
http://twitter.com/BretWortman
On Apr 28, 2014, at 8:20 AM, Dmitri Pal d...@redhat.com wrote:
On 04/28/2014 08:06 AM, Bret Wortman wrote:
Not to be thick, but what's the best way to check the DS instance for
a pki entry?
I do not remember the exact path and I do not have an instance handy.
Something like /var/lib/dirsrv/PKI, do not want to mislead you.
On 04/28/2014 07:57 AM, Dmitri Pal wrote:
On 04/28/2014 07:52 AM, Bret Wortman wrote:
I'm trying to stand up a new ipa server on a clean box, and I keep getting
this error so _something_ is amiss but I'm not sure what:
:
Configuring certificate server (pki-tomcatd): Estimated time 3 minutes 30
seconds
[1/22]: creating certificate server user
[2/22]: configuring certificate server instance
ipa: CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit status
1
Configuration of CA failed
#
In the /var/log/ipaserver-install.log, I see this:
:
:
Installing CA into /var/lib/pki/pki-tomcat.
Installation failed.
2014-04-28T11:43:46Z DEBUG stderr=pkispawn : ERROR PKI
subsystem 'CA' for instance 'pki-tomcat' already exists!
2014-04-28T11:432:46Z CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpX8RW20' returned non-zero exit status
1
2014-04-28T11:43:46Z DEBUG File
/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py, line
622, in run_script
return_value = main_function()
File /usr/sbin/ipa-server-install, line 1074, in main
dm_password, subject_base=options.subject)
File /usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py,
line 478, in configure_instance
self.start_creation(runtime=210)
File /usr/lib/python2.7/site-packages/ipaserver/isntall/service.py,
line 364, in start_creation
method()
File /usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py,
line 604, in __spawn_instance
raise RUntimeError('Configuration of CA failed')
:
:
So it looks like somehow this has gotten configured already. Possibly
Puppet copied over something it shouldn't have. What do I need to remove
to make this step work without removing so much that I render something
inoperable?
Run uninstall several times. Each time uninstall might clean next portion
and untangle things so trying to do it several times pays off.
Then check if there is a DS instance for PKI. If there is remove it and try
again.
--
Bret Wortman
mime-attachment.png
http://damascusgrp.com/
http://about.me/wortmanbret
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
--
Thank you,
Dmitri Pal
Sr. Engineering Manager IdM portfolio
Red Hat, Inc.
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
--
Thank you,
Dmitri Pal
Sr. Engineering Manager IdM portfolio
Red Hat, Inc.
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
smime.p7s
Description: S/MIME cryptographic signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
On 04/28/2014 10:48 AM, Rob Crittenden wrote: Bret Wortman wrote: On 04/28/2014 10:21 AM, Bret Wortman wrote: On 04/28/2014 08:33 AM, Petr Viktorin wrote: According to the error you're getting, there is a CA instance already installed. After uninstalling IPA, destroy it with: pkidestroy -s CA -i pki-tomcat I tried, this, but no joy. # pkidestroy -s CA -i pki-tomcat Loading deployment configuration from /var/lib/pki/pki-tomcat /ca/registry/ca/deployment.cfg. Uninstalling CA from /var/lib/pki/pki-tomcat. pkidestroy : WARNING ... this 'CA' entry will NOT be deleted from security domain 'unknown'! pkidestroy : ERROR ... No security domain defined. If this is an unconfigured instance, then that is OK. Otherwise, manually delete the entry from the security domain master. Uninstallation complete. # And then when I tried to run ipa-server-install, I got the same error again. I may just wipe the box and start over. It might take less time overall. Bret This, BTW, is on F20 using freeipa 3.3.4-3 and pki-ca 10.1.1-1 (also dogtag-10.1.1-1). From the ipa-server installation output the error looks the same, but the underlying error should be different when there isn't already a PKI instance. If the PKI installer fails early enough we don't record that it was installed which is why ipa-server-install --uninstall doesn't remove it. We have a ticket open for this. rob So is there a recommended way to clean it up and get it working? smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
On 04/28/2014 11:08 AM, Bret Wortman wrote: On 04/28/2014 10:48 AM, Rob Crittenden wrote: Bret Wortman wrote: On 04/28/2014 10:21 AM, Bret Wortman wrote: On 04/28/2014 08:33 AM, Petr Viktorin wrote: According to the error you're getting, there is a CA instance already installed. After uninstalling IPA, destroy it with: pkidestroy -s CA -i pki-tomcat I tried, this, but no joy. # pkidestroy -s CA -i pki-tomcat Loading deployment configuration from /var/lib/pki/pki-tomcat /ca/registry/ca/deployment.cfg. Uninstalling CA from /var/lib/pki/pki-tomcat. pkidestroy : WARNING ... this 'CA' entry will NOT be deleted from security domain 'unknown'! pkidestroy : ERROR ... No security domain defined. If this is an unconfigured instance, then that is OK. Otherwise, manually delete the entry from the security domain master. Uninstallation complete. # And then when I tried to run ipa-server-install, I got the same error again. I may just wipe the box and start over. It might take less time overall. Bret This, BTW, is on F20 using freeipa 3.3.4-3 and pki-ca 10.1.1-1 (also dogtag-10.1.1-1). From the ipa-server installation output the error looks the same, but the underlying error should be different when there isn't already a PKI instance. If the PKI installer fails early enough we don't record that it was installed which is why ipa-server-install --uninstall doesn't remove it. We have a ticket open for this. rob So is there a recommended way to clean it up and get it working? Never mind; I found the bug (953488) which said to: # pkidestroy -s CA -i pki-tomcat ERROR: PKI instance '/var/lib/pki/pki-tomcat' does NOT exist! # rm -rf /var/log/pki/pki-tomcat # rm -rf /etc/sysconfig/pki-tomcat # rm -rf /etc/sysconfig/pki/tomcat/pki-tomcat # rm -rf /var/lib/pki/pki-tomcat # rm -rf /etc/pki/pki-tomcat # ipa-server-install --uninstall And re-run installation. This didn't work for me. Was there another bug that I missed? smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
On 04/28/2014 11:17 AM, Rob Crittenden wrote:
Bret Wortman wrote:
So is there a recommended way to clean it up and get it working?
Re-run pkidestroy, then if the subsequent IPA install fails closely
examine the logs to determine the reason. The problem in cases like
this is that the first install fails and subsequent installs mask the
original failure with this PKI re-install failure.
rob
Okay, here's the log from when it starts configuring PKI:
2014-04-28T15:23:45Z DEBUG [2/22]: configuring certificate server instance
2014-04-28T15:23:45Z DEBUG Contents of pkispawn configuration file
(/tmp/tmpdCm6rt):
[CA]
pki_security_domain_name = IPA
pki_enable_proxy = True
pki_restart_configured_instance = False
pki_backup_keys = True
pki-backup_password =
pki_client_database_dir = /tmp/tmp-rVoTR2
pki_client_database_password =
pki_client_database_purge = False
pki_client_pkcs12_password =
pki_admin_name = admin
pki_admin_uid = admin
pki_admin_email = root@localhost
pki_admin_password =
pki_admin_nickname = ipa-ca-agent
pki_admin_subject_dn = cn=ipa-ca-agent,O=FOO.NET
pki_client_admin_cert_p12 = /root/ca-agent.p12
pki_ds_ldap_port = 389
pki_ds_password =
pki_ds_base_dn = o=ipaca
pki_ds_database = ipaca
pki_subsystem_subject+dn = cn=CA Subsystem,O=FOO.NET
pki_ocsp_signing_subject_dn = cn=OCSP Subsystem,O=FOO.NET
pki_ssl_server_subject_dn = cn=zsipa.foo.net,O=FOO.NET
pki_audit_signing_subject_dn = cn=CA Audit,O=FOO.NET
pki_ca_signing_subject_dn = cn-Certificate Authority,O=FOO.NET
pki_subsystem_nickname = subsystemCert cert-pki-ca
pki_ocsp_signing_nickname = ocspSigningCert cert-pki-ca
pki_ssl_server_nickname = Server-Cert cert-pki-ca
pki_audit_signing_nickname = auditSigningCert cert-pki-ca
pki_ca_signing_nickname = caSigningCert cert-pki-ca
2014-04-28T15:23:45Z DEBUG Starting external process
2014-04-28T15:23:45Z DEBUG args=/usr/sbin/pkispawn -s CA -f /tmp/tmpdCm6rt
2014-04-28T15:23:45Z DEBUG Process finished, return code=1
2014-04-28T15:23:45Z DEBUG stdout=Loading deployment configuration from
/tmp/tmpdCm6rt.
Installing CA into /var/lib/pki/pki-tomcat.
Storing deployment configuration into
/etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg
Installation failed.
2014-04-28T15:24:46Z DEBUG stderr=pkispawn : ERROR ... server
failed to restart
2014-04-28T15:24:46Z CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpdCm6rt' returned non-zero exit status 1
2014-04-28T15:24:46Z DEBUG File
/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py,
line 622, in run_script
return_value = main_function()
File /usr/sbin/ipa-server-install, line 1074, in main
dm_password, subject_base=options.subject)
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
478, in configure_instance
self.start_creation(runtime=210)
File /usr/lib/python2.7/site-packages/ipaserver/isntall/service.py,
line 364, in start_creation
method()
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
604, in __spawn_instance
raise RUntimeError('Configuration of CA failed')
2014-04-28T15:24:46Z DEBUG The ipa-server-install command failed,
exception: RuntimeError: Configuration of CA failed
And that's the end of the log. Nothing here looks terribly informative
to me, and this is what the log looks like every time I look at it.
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Error creating new freeipa-server
On 04/28/2014 11:52 AM, Rob Crittenden wrote:
Bret Wortman wrote:
On 04/28/2014 11:17 AM, Rob Crittenden wrote:
Bret Wortman wrote:
So is there a recommended way to clean it up and get it working?
Re-run pkidestroy, then if the subsequent IPA install fails closely
examine the logs to determine the reason. The problem in cases like
this is that the first install fails and subsequent installs mask the
original failure with this PKI re-install failure.
rob
Okay, here's the log from when it starts configuring PKI:
2014-04-28T15:23:45Z DEBUG [2/22]: configuring certificate server
instance
2014-04-28T15:23:45Z DEBUG Contents of pkispawn configuration file
(/tmp/tmpdCm6rt):
[CA]
pki_security_domain_name = IPA
pki_enable_proxy = True
pki_restart_configured_instance = False
pki_backup_keys = True
pki-backup_password =
pki_client_database_dir = /tmp/tmp-rVoTR2
pki_client_database_password =
pki_client_database_purge = False
pki_client_pkcs12_password =
pki_admin_name = admin
pki_admin_uid = admin
pki_admin_email = root@localhost
pki_admin_password =
pki_admin_nickname = ipa-ca-agent
pki_admin_subject_dn = cn=ipa-ca-agent,O=FOO.NET
pki_client_admin_cert_p12 = /root/ca-agent.p12
pki_ds_ldap_port = 389
pki_ds_password =
pki_ds_base_dn = o=ipaca
pki_ds_database = ipaca
pki_subsystem_subject+dn = cn=CA Subsystem,O=FOO.NET
pki_ocsp_signing_subject_dn = cn=OCSP Subsystem,O=FOO.NET
pki_ssl_server_subject_dn = cn=zsipa.foo.net,O=FOO.NET
pki_audit_signing_subject_dn = cn=CA Audit,O=FOO.NET
pki_ca_signing_subject_dn = cn-Certificate Authority,O=FOO.NET
pki_subsystem_nickname = subsystemCert cert-pki-ca
pki_ocsp_signing_nickname = ocspSigningCert cert-pki-ca
pki_ssl_server_nickname = Server-Cert cert-pki-ca
pki_audit_signing_nickname = auditSigningCert cert-pki-ca
pki_ca_signing_nickname = caSigningCert cert-pki-ca
2014-04-28T15:23:45Z DEBUG Starting external process
2014-04-28T15:23:45Z DEBUG args=/usr/sbin/pkispawn -s CA -f
/tmp/tmpdCm6rt
2014-04-28T15:23:45Z DEBUG Process finished, return code=1
2014-04-28T15:23:45Z DEBUG stdout=Loading deployment configuration from
/tmp/tmpdCm6rt.
Installing CA into /var/lib/pki/pki-tomcat.
Storing deployment configuration into
/etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg
Installation failed.
2014-04-28T15:24:46Z DEBUG stderr=pkispawn : ERROR ... server
failed to restart
2014-04-28T15:24:46Z CRITICAL failed to configure ca instance Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpdCm6rt' returned non-zero exit
status 1
2014-04-28T15:24:46Z DEBUG File
/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py,
line 622, in run_script
return_value = main_function()
File /usr/sbin/ipa-server-install, line 1074, in main
dm_password, subject_base=options.subject)
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
478, in configure_instance
self.start_creation(runtime=210)
File /usr/lib/python2.7/site-packages/ipaserver/isntall/service.py,
line 364, in start_creation
method()
File
/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py, line
604, in __spawn_instance
raise RUntimeError('Configuration of CA failed')
2014-04-28T15:24:46Z DEBUG The ipa-server-install command failed,
exception: RuntimeError: Configuration of CA failed
And that's the end of the log. Nothing here looks terribly informative
to me, and this is what the log looks like every time I look at it.
The error is different whether there is an existing PKI instance or not.
The next set of logs to look at are in /var/log/pki. It says there is
a startup failure so I'd start with
*/var/log/pki/pki-tomcat/catalina.out* . Also interesting may be the
pki-ca-spawn and debug logs found within that directory structure.
I'd also look for SELinux errors with ausearch -m AVC -ts recent
This did the trick. Something was hanging out on port 8443, though
neither lsof nor netstat would show me what it was. I rebooted the
server and then it proceeded past this without a hiccup.
Thanks, Rob and everyone else for helping me navigate the logs!
Bret
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Can't use ipa commands on brand new ipa server instance
I just got a new ipa server instantiated and haven't actually installed any users or hosts on it yet. No replicas. No migrated data. Yet when I run any ipa commands from the command line, it behaves exactly as our older, troubled servers do and exits the login session immediately, whether I'm connected at the console or via ssh. Further, when I run strace to try to capture what might be going on, the behavior stops. Script also prevents commands from exiting, but this is really disconcerting. I was chalking this up to the fact that our database had become corrupted by our replication problems, but now I'm thinking it might be environmental, though our original IPA servers are running F18 and this new instance is F20. I need some stability here, and CLI is part of that. What might be causing the CLI to not work at all when coupled to a TTY device, as that seems to be the critical piece? Could this be related to the servers being VMs? -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Can't use ipa commands on brand new ipa server instance
On 04/28/2014 01:19 PM, Bret Wortman wrote: I just got a new ipa server instantiated and haven't actually installed any users or hosts on it yet. No replicas. No migrated data. Yet when I run any ipa commands from the command line, it behaves exactly as our older, troubled servers do and exits the login session immediately, whether I'm connected at the console or via ssh. Further, when I run strace to try to capture what might be going on, the behavior stops. Script also prevents commands from exiting, but this is really disconcerting. I was chalking this up to the fact that our database had become corrupted by our replication problems, but now I'm thinking it might be environmental, though our original IPA servers are running F18 and this new instance is F20. I need some stability here, and CLI is part of that. What might be causing the CLI to not work at all when coupled to a TTY device, as that seems to be the critical piece? Could this be related to the servers being VMs? BTW, we have this running on F20 on a different network and it works just fine. The network on which the failures are occurring isn't internet-connected; is there something that's trying to connect back to redhat? smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Can't use ipa commands on brand new ipa server instance
bash. On 04/28/2014 01:32 PM, Simo Sorce wrote: On Mon, 2014-04-28 at 13:25 -0400, Bret Wortman wrote: On 04/28/2014 01:19 PM, Bret Wortman wrote: I just got a new ipa server instantiated and haven't actually installed any users or hosts on it yet. No replicas. No migrated data. Yet when I run any ipa commands from the command line, it behaves exactly as our older, troubled servers do and exits the login session immediately, whether I'm connected at the console or via ssh. Further, when I run strace to try to capture what might be going on, the behavior stops. Script also prevents commands from exiting, but this is really disconcerting. I was chalking this up to the fact that our database had become corrupted by our replication problems, but now I'm thinking it might be environmental, though our original IPA servers are running F18 and this new instance is F20. I need some stability here, and CLI is part of that. What might be causing the CLI to not work at all when coupled to a TTY device, as that seems to be the critical piece? Could this be related to the servers being VMs? BTW, we have this running on F20 on a different network and it works just fine. The network on which the failures are occurring isn't internet-connected; is there something that's trying to connect back to redhat? no. What shell do you use ? Simo. smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Can't use ipa commands on brand new ipa server instance
On 04/28/2014 01:53 PM, Simo Sorce wrote: On 04/28/2014 01:32 PM, Simo Sorce wrote: On Mon, 2014-04-28 at 13:25 -0400, Bret Wortman wrote: On 04/28/2014 01:19 PM, Bret Wortman wrote: I just got a new ipa server instantiated and haven't actually installed any users or hosts on it yet. No replicas. No migrated data. Yet when I run any ipa commands from the command line, it behaves exactly as our older, troubled servers do and exits the login session immediately, whether I'm connected at the console or via ssh. Further, when I run strace to try to capture what might be going on, the behavior stops. Script also prevents commands from exiting, but this is really disconcerting. I was chalking this up to the fact that our database had become corrupted by our replication problems, but now I'm thinking it might be environmental, though our original IPA servers are running F18 and this new instance is F20. I need some stability here, and CLI is part of that. What might be causing the CLI to not work at all when coupled to a TTY device, as that seems to be the critical piece? Could this be related to the servers being VMs? BTW, we have this running on F20 on a different network and it works just fine. The network on which the failures are occurring isn't internet-connected; is there something that's trying to connect back to redhat? no. What shell do you use ? On Mon, 2014-04-28 at 13:43 -0400, Bret Wortman wrote: bash. Does it make any difference if you redirect stdin before calling the command ? Simo. No, I found the problem. A power user had written a bash function that redefined ipa and dropped it into /etc/profile.d. We're about to have a little chat. smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Badly corrupted IPA
My IPA corruption continues and I'm afraid I'm going to have to
recreate it from scratch since no reasonable means of backup exists
(which I understand -- that's not my complaint).
Here's what I'm facing:
# script -c 'ipa host-find mw79.damascusgrp.com'
Script started, file is typescript
--
1 host matched
--
Host name: mw79.damascusgrp.com
Principal name: host/mw79.damascusgrp@damascusgrp.com
Password: False
Member of host-groups: allow_all_hosts
Indirect Member of HBAC rule: allow_all_users_services
Keytab: False
SSH public key fingerprint: [snip] (ssh-dss)
Number of entries returned 1
Script done, file is typescript
# script -c 'ipa host-del mw79.damascusgrp.com'
Script started, file is typescript
ipa: ERROR: mw79.damascusgrp.com: host not found
Script done, file is typescript
#
I had unenrolled this host and was attempting to re-enroll it,
and this is preventing its re-enrollment. Any ideas of how to force
deletion of this host entry? I'm not LDAP savvy enough to just go in
and start whacking LDAP entries myself, and given that my IPA
database has gotten corrupted enough that no IPA CLI command can run
without being wrapped in "script" or "strace" or similar, I'm
hesitant to go too far. This machine, however, is my program
manager's workstation, so it's pretty important to get back up and
running.
Thanks,
--
Bret Wortman
http://damascusgrp.com/
http://about.me/wortmanbret
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Badly corrupted IPA
BTW, this also fails when using the web UI -- I can see the entry
but not delete it.
On 03/27/2014 09:02 AM, Bret Wortman
wrote:
My IPA corruption continues and I'm afraid I'm going to have to
recreate it from scratch since no reasonable means of backup
exists (which I understand -- that's not my complaint).
Here's what I'm facing:
# script -c 'ipa host-find mw79.damascusgrp.com'
Script started, file is typescript
--
1 host matched
--
Host name: mw79.damascusgrp.com
Principal name: host/mw79.damascusgrp@damascusgrp.com
Password: False
Member of host-groups: allow_all_hosts
Indirect Member of HBAC rule: allow_all_users_services
Keytab: False
SSH public key fingerprint: [snip] (ssh-dss)
Number of entries returned 1
Script done, file is typescript
# script -c 'ipa host-del mw79.damascusgrp.com'
Script started, file is typescript
ipa: ERROR: mw79.damascusgrp.com: host not found
Script done, file is typescript
#
I had unenrolled this host and was attempting to re-enroll
it, and this is preventing its re-enrollment. Any ideas of how to
force deletion of this host entry? I'm not LDAP savvy enough to
just go in and start whacking LDAP entries myself, and given that
my IPA database has gotten corrupted enough that no IPA CLI
command can run without being wrapped in "script" or "strace" or
similar, I'm hesitant to go too far. This machine, however, is my
program manager's workstation, so it's pretty important to get
back up and running.
Thanks,
--
Bret Wortman
http://damascusgrp.com/
http://about.me/wortmanbret
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Badly corrupted IPA
That worked like a champ. As always. Thanks, Rob. Bret On 03/27/2014 10:08 AM, Rob Crittenden wrote: Bret Wortman wrote: BTW, this also fails when using the web UI -- I can see the entry but not delete it. It sounds like you have a replication conflict entry. Try this search: $ ldapsearch -Y GSSAPI -b cn=computers,cn=accounts,dc=example,dc=com fdqdn=myhost.example.com You'll probably get something with a dn that includes a nsuniqueid in it. That is the conflict entry. IPA can find the host because it searches by fqdn too, but it deletes by generating the direct DN and since it doesn't match, no delete. You can delete the wayward entry using ldapdelete. rob On 03/27/2014 09:02 AM, Bret Wortman wrote: My IPA corruption continues and I'm afraid I'm going to have to recreate it from scratch since no reasonable means of backup exists (which I understand -- that's not my complaint). Here's what I'm facing: # script -c 'ipa host-find mw79.damascusgrp.com' Script started, file is typescript -- 1 host matched -- Host name: mw79.damascusgrp.com Principal name: host/mw79.damascusgrp@damascusgrp.com Password: False Member of host-groups: allow_all_hosts Indirect Member of HBAC rule: allow_all_users_services Keytab: False SSH public key fingerprint: [snip] (ssh-dss) Number of entries returned 1 Script done, file is typescript # script -c 'ipa host-del mw79.damascusgrp.com' Script started, file is typescript ipa: ERROR: mw79.damascusgrp.com: host not found Script done, file is typescript # I had unenrolled this host and was attempting to re-enroll it, and this is preventing its re-enrollment. Any ideas of how to force deletion of this host entry? I'm not LDAP savvy enough to just go in and start whacking LDAP entries myself, and given that my IPA database has gotten corrupted enough that no IPA CLI command can run without being wrapped in script or strace or similar, I'm hesitant to go too far. This machine, however, is my program manager's workstation, so it's pretty important to get back up and running. Thanks, -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] Password issues
Strange behavior now with our passwords (and we still haven't solved
our problem with the "ipa" command, but at least with script, we
have a workaround):
I noticed yesterday morning that my password, which has the
following policy, was going to expire in 3 days so I changed it.
Max lifetime (days) : 0
Min lifetime (hours) : 0
History size (number of passwords): 0
Character classes: 2
Min length: 8
Max failures: 4
Failure reset interval (seconds): 60
Lockout duration (seconds): 60
The IPA web UI immediately began reporting in red that "Your
password expires in -1 days."
This morning, I ran "kinit":
$ kinit
Password for br...@damascusgrp.com:
Password expired. You must change it now.
Enter new password:
Enter it again:
Warning: Your password wille xpire in less than one hour on
Thu 06 Mar 2014 06:45:48 AM EST
$
What's up? I'd like to solve this before it bites any of my users,
though most have a policy that looks more like this:
Max lifetime (days) : 180
Min lifetime (hours) : 1
History size (number of passwords): 0
Character classes: 2
Min length: 8
Max failures: 6
Failure reset interval (seconds): 60
Lockout duration (seconds): 600
--
Bret Wortman
http://damascusgrp.com/
http://about.me/wortmanbret
smime.p7s
Description: S/MIME Cryptographic Signature
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Password issues
Is there a way to set a password to not expire? I thought I read somewhere that 0 did that, but apparently not. On 03/06/2014 07:55 AM, Sumit Bose wrote: On Thu, Mar 06, 2014 at 07:39:15AM -0500, Bret Wortman wrote: Strange behavior now with our passwords (and we still haven't solved our problem with the ipa command, but at least with script, we have a workaround): I noticed yesterday morning that my password, which has the following policy, was going to expire in 3 days so I changed it. Max lifetime (days) : 0 I think the behaviour is expected with this maximal lifetime. bye, Sumit Min lifetime (hours) : 0 History size (number of passwords): 0 Character classes: 2 Min length: 8 Max failures: 4 Failure reset interval (seconds): 60 Lockout duration (seconds): 60 The IPA web UI immediately began reporting in red that Your password expires in -1 days. This morning, I ran kinit: $ kinit Password for br...@damascusgrp.com: Password expired. You must change it now. Enter new password: Enter it again: Warning: Your password wille xpire in less than one hour on Thu 06 Mar 2014 06:45:48 AM EST $ What's up? I'd like to solve this before it bites any of my users, though most have a policy that looks more like this: Max lifetime (days) : 180 Min lifetime (hours) : 1 History size (number of passwords): 0 Character classes: 2 Min length: 8 Max failures: 6 Failure reset interval (seconds): 60 Lockout duration (seconds): 600 -- *Bret Wortman* http://damascusgrp.com/ http://about.me/wortmanbret ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
