Hi Rob:

First I wanted to thank you for all of your valuable input/tips. As you well 
know, everything about certs, certmonger, dogtag and FreeIPA can get very 
complicated - there’s no easy answer, so many things can go wrong :) 

But, your answers to my questions got me thinking, gave me some clues, pointed 
me in the right direction.

I wanted to take the time to specifically thank you because these concepts have 
mystified me for quite a while, our FreeIPA system has been running for more 
than a year with everything regarding certs kinda wacky and with me just 
praying that that fact didn’t crash everything and make the most important 
function for us (ssh, sssd, authentication, sso) stop working.

With your help I have certainly not become an expert but have gone from pretty 
much clueless to having somewhat of a clue :) That’s progress !!

My issue with the CA certs themselves is solved thanks to you pointing out the 
issue with creating replicas in 3.0 which has been fixed in 3.3 - the issue 
that can be solved by manually exporting a new cacert.p12 file and boom, new 
replicas created with expired certs issue solved.

And then there was the issue of “sec error legacy database” which would 
manifest itself in various forms and can be caused by many things - it is 
temporarily solved by restarting httpd but then just comes right back. 

Based on your input I started looking at the certs/certmonger/getcert list - on 
all my nodes/hosts and noticed that many of them had bogus certs with principal 
names pointed at hosts that no longer existed. No other way to describe them 
other than WTF !!.

My theory now is that all the nodes calling in to the CA with all those bogus 
certs were just overloading the CA and so after restarting httpd, it would 
temporarily clear up until all the nodes starting calling in to the CA again - 
or something like that.

Anyways, Ansible to the rescue….

I exported a list of hosts from my IPA system, that became my Ansible inventory 
file.

Now, throw together a quick playbook to look at every host, identify the bogus 
cert or certs and tell certmonger to stop tracking them.

The simple Ansible playbook follows here.

Run that against all hosts and bingo !!!  - my httpd logs on the CA are no 
longer getting spammed with bogus cert requests, “sec error legacy database” 
errors are not happening, etc , etc.

In short, my FreeIPA CA situation is now, I hope and pray, fairly stable.

So HUGE shout out to you Rob !!!


---
- hosts: ipa-hosts
  gather_facts: False

  tasks:

  - name: get request id
    shell: ipa-getcert list -r | gawk -F\' '/Request/ {print $2}'
    register: my_id

  #- debug: var=my_id

  - name: kill bad certs
    shell: ipa-getcert stop-tracking -i {{ item }}
    with_items: "{{ my_id.stdout_lines }}"


 <http://www.placeiq.com/> <http://www.placeiq.com/> <http://www.placeiq.com/>  
Jim Richard      <https://twitter.com/placeiq> <https://twitter.com/placeiq> 
<https://twitter.com/placeiq>       <https://www.facebook.com/PlaceIQ> 
<https://www.facebook.com/PlaceIQ>   <https://www.linkedin.com/company/placeiq> 
<https://www.linkedin.com/company/placeiq>
SYSTEM ADMINISTRATOR III
(646) 338-8905  

 
<http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/>
 
<http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
 
<http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
 
<http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
 
<http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
 
<http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/>
 
<http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
 
<http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
 
<http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
 
<http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
 
<http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
 
<http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>


> On Sep 30, 2016, at 4:53 AM, Rob Crittenden <rcrit...@redhat.com> wrote:
> 
> Jim Richard wrote:
>> Can I and how…
>> 
>> delete all certs for all hosts
>> 
>> I mean, we only use FreeIPA for user login/sssd
>> 
>> That said, do we even need those certs?
> 
> There is no simple answer, really.
> 
> Yes, you can deleted all certs for all hosts (not recommended as some of 
> those are for IPA services). I doubt it would do anything positive and if the 
> certificate is tracked by certmonger on the client it would eventually renew.
> 
> Do you need the certs? Only you would know that, but chances are the vast 
> majority aren't being used.
> 
> In 3.0 when a client is registered a host certificate is obtained for it. 
> This certificate was never used and in 4.something it isn't requested at all 
> unless an option is passed to ipa-client-install.
> 
> rob
> 
>> 
>> 
>> 
>> <http://www.placeiq.com/><http://www.placeiq.com/><http://www.placeiq.com/>
>> Jim Richard
>> <https://twitter.com/placeiq><https://twitter.com/placeiq><https://twitter.com/placeiq>
>> <https://www.facebook.com/PlaceIQ><https://www.facebook.com/PlaceIQ>
>> <https://www.linkedin.com/company/placeiq><https://www.linkedin.com/company/placeiq>
>> SYSTEM ADMINISTRATOR III
>> /(646) 338-8905 /
>> 
>> 
>> <http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-a!
> dvertising
> -initiative-nai-as-100th-member/>PlaceIQ:Location
>> Data Accuracy
>> <http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>
>> 
>> 
>> 
>>> On Sep 29, 2016, at 8:53 PM, Jim Richard <jrich...@placeiq.com
>>> <mailto:jrich...@placeiq.com>> wrote:
>>> 
>>> another interesting thing, my httpd/error_logs are constantly getting
>>> spammed with: (I removed the stuff between the single quotes)
>>> 
>>> Notice those names don’t match, should they?
>>> 
>>> Me thinks not since those “principal=“ items are ALMOST all hosts that
>>> no longer exist in the FreeIPA system. I rare few do exist.
>>> 
>>> So, that’s weird :)
>>> 
>>> [Thu Sep 29 20:44:59 2016] [error] ipa: INFO:
>>> host/aerospike-cl1-203.nym1.placeiq....@placeiq.net
>>> <mailto:host/aerospike-cl1-203.nym1.placeiq....@placeiq.net>:
>>> cert_request(u’…………………..',
>>> principal=u'host/sbtt-nyc1-028.thum01.nym1.placeiq....@placeiq.net
>>> <mailto:principal=u'host/sbtt-nyc1-028.thum01.nym1.placeiq....@placeiq.net>',
>>> add=True): CertificateOperationError
>>> 
>>> [Thu Sep 29 20:45:06 2016] [error] ipa: INFO:
>>> host/aerospike-cl2-210.nym1.placeiq....@placeiq.net
>>> <mailto:host/aerospike-cl2-210.nym1.placeiq....@placeiq.net>:
>>> cert_request(u’…………………..',
>>> principal=u'host/017.prod07.nym1.placeiq....@placeiq.net
>>> <mailto:principal=u'host/017.prod07.nym1.placeiq....@placeiq.net>',
>>> add=True): CertificateOperationError
>>> 
>>> [Thu Sep 29 20:45:09 2016] [error] ipa: INFO:
>>> host/adsgateway-14.nym1.placeiq....@placeiq.net
>>> <mailto:host/adsgateway-14.nym1.placeiq....@placeiq.net>:
>>> cert_request(u’……………………...',
>>> principal=u'host/025.prod07.nym1.placeiq....@placeiq.net
>>> <mailto:principal=u'host/025.prod07.nym1.placeiq....@placeiq.net>',
>>> add=True): CertificateOperationError
>>> 
>>> [Thu Sep 29 20:45:29 2016] [error] ipa: INFO:
>>> host/ttsandbox-022.nym1.placeiq....@placeiq.net
>>> <mailto:host/ttsandbox-022.nym1.placeiq....@placeiq.net>:
>>> cert_request(u’……………………….',
>>> principal=u'host/sbtt-nyc1-022.thum01.nym1.placeiq....@placeiq.net
>>> <mailto:principal=u'host/sbtt-nyc1-022.thum01.nym1.placeiq....@placeiq.net>',
>>> add=True): CertificateOperationError
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> <http://www.placeiq.com/><http://www.placeiq.com/><http://www.placeiq.com/>
>>> Jim Richard
>>> <https://twitter.com/placeiq><https://twitter.com/placeiq><https://twitter.com/placeiq>
>>> <https://www.facebook.com/PlaceIQ><https://www.facebook.com/PlaceIQ>
>>> <https://www.linkedin.com/company/placeiq><https://www.linkedin.com/company/placeiq>
>>> SYSTEM ADMINISTRATOR III
>>> /(646) 338-8905 /
>>> 
>>> 
>>> <http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/><http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/><http://placeiq.com/2016/04/13/placeiq-joins-the-network-!
> advertisin
> g-initiative-nai-as-100th-member/>PlaceIQ:Location
>>> Data Accuracy
>>> <http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>
>>> 
>>> 
>>> 
>>>> On Sep 29, 2016, at 8:11 AM, Rob Crittenden <rcrit...@redhat.com
>>>> <mailto:rcrit...@redhat.com>> wrote:
>>>> 
>>>> Natxo Asenjo wrote:
>>>>> hi Jim,
>>>>> 
>>>>> On Thu, Sep 29, 2016 at 7:37 AM, Jim Richard <jrich...@placeiq.com
>>>>> <mailto:jrich...@placeiq.com>
>>>>> <mailto:jrich...@placeiq.com>> wrote:
>>>>> 
>>>>>   Thanks Rob, that worked.
>>>>> 
>>>>>   Still on the subject of certs, any idea how to solve this error:
>>>>> 
>>>>>   Certificate format error: (SEC_ERROR_LEGACY_DATABASE) The
>>>>>   certificate/key database is in an old, unsupported format.
>>>>> 
>>>>>   I see that in the gui when querying hosts as well as from cli when I
>>>>>   ipa-show or ipa-find
>>>>> 
>>>>> 
>>>>> I have had this too, and we did not find a solution (search my recent
>>>>> posts on the archives). As a workaround I have created replicas and
>>>>> decommissioned the older replicas.
>>>> 
>>>> On the one hand I'm glad this fixed it for you. On the other it is a
>>>> rather unsatisfying answer. Unfortunately NSS doesn't always provide
>>>> the most context with its error messages. This error is usually seen
>>>> when one tries to open a non-existent database, which in this case is
>>>> a very strange thing, especially since it goes from working to
>>>> non-working in the same apache process over a few minutes.
>>>> 
>>>> I'm not sure how I'd troubleshoot this if it were easily
>>>> reproducible. I suspect we'd need to figure out which database cannot
>>>> be found (most likely /etc/httpd/alias) and go from there. An strace
>>>> is a brute-force way to see the file open but finding the right
>>>> process to attach to is a bit of an art.
>>>> 
>>>> rob
>>>> 
>>>> --
>>>> Manage your subscription for the Freeipa-users mailing list:
>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>> Go to http://freeipa.org for more info on the project
>>> 
>> 
> 

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to