On 11/12/2015 04:51 PM, Terry John wrote:


I got a core dump of certmonger failing user abrt but it's huge. Is there any 
particular part that would be useful.

CCing Nalin and David for the core dump. More below.


On 11/12/2015 02:17 PM, Terry John wrote:
I had a working freeipa setup on a CentOS release 6.7 machine.  All was well 
until I did a yum update. Now I have multiple issue apparently based around the 
CMS (Service Unavailable) issue.
My current version of ipa-server is 3.0.0-47
Certmonger crashes with a segmentation fault at boot time and crashes every 
time I try to restart it when ipa is running.

It of course should not crash, it would be useful to have a backtrace from the 
core file that was generated.
Here is the backtrace of the core file:
{   "signal": 11
,   "executable": "/usr/sbin/certmonger"
,   "stacktrace":
       [ {   "crash_thread": true
         ,   "frames":
               [ {   "address": 140527158519285
                 ,   "build_id": "87a19a61dc011579f3e25de3ca9778c6fd9e4547"
                 ,   "build_id_offset": 1222133
                 ,   "function_name": "__strstr_sse42"
                 ,   "file_name": "/lib64/libc.so.6"
                 }
               , {   "address": 140527209363149
                 ,   "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
                 ,   "build_id_offset": 141005
                 ,   "file_name": "/usr/sbin/certmonger"
                 }
               , {   "address": 140527209301676
                 ,   "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
                 ,   "build_id_offset": 79532
                 ,   "file_name": "/usr/sbin/certmonger"
                 }
               , {   "address": 140527209287550
                 ,   "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
                 ,   "build_id_offset": 65406
                 ,   "file_name": "/usr/sbin/certmonger"
                 }
               , {   "address": 140527209291166
                 ,   "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
                 ,   "build_id_offset": 69022
                 ,   "file_name": "/usr/sbin/certmonger"
                 }
               , {   "address": 140527196303038
                 ,   "build_id": "4135efbfc51bb80e4945275a6e6ba10e9d8a2a11"
                 ,   "build_id_offset": 36542
                 ,   "file_name": "/usr/lib64/libtevent.so.0"
                 }
               , {   "address": 140527196295910
                 ,   "build_id": "4135efbfc51bb80e4945275a6e6ba10e9d8a2a11"
                 ,   "build_id_offset": 29414
                 ,   "file_name": "/usr/lib64/libtevent.so.0"
                 }
               , {   "address": 140527196279965
                 ,   "build_id": "4135efbfc51bb80e4945275a6e6ba10e9d8a2a11"
                 ,   "build_id_offset": 13469
                 ,   "function_name": "_tevent_loop_once"
                 ,   "file_name": "/usr/lib64/libtevent.so.0"
                 }
               , {   "address": 140527209278079
                 ,   "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
                 ,   "build_id_offset": 55935
                 ,   "function_name": "main"
                 ,   "file_name": "/usr/sbin/certmonger"
                 } ]
         } ]
}

In /var/log/messages I get
freeipasvr kernel: certmonger[2611] general protection ip:7fb487fed5f5 
sp:7ffd9df46898 error:0 in libc-2.12.so[7fb487ec3000+18a000]

This is the first error I get in /var/log/httpd/error_log when I try to delete 
a host
[error] ipa: ERROR: ipaserver.plugins.dogtag.ra.get_certificate(): Unable to 
communicate with CMS (Service Unavailable)

If I stop ipa the start certmonger it starts ok and continues to run when I start ipa 
again but as soon as any requests are made like "getcert list" then it crashes 
again.
With certmonger still running I can do a request

# ipa cert-status
Request id: 20140417164153
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (Service Unavailable) # service certmonger status
certmonger (pid  3030) is running...

It looks like PKI cannot be contacted. I would recommend checking 
/var/log/httpd/error_log, it may have more details. I would also recommend checking 
"ipa cert-show 1", it will probably fail with the same bug.
Yes ipa cert-show 1 does show the same thing
# ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: Unable to communicate 
with CMS (Service Unavailable)

Next steps may include checking that dogtag service really runs, there is no 
SELinux AVC. If neither of this helps, you can check PKI logs /var/log/pki... 
to see what went wrong.
I'm sure SELinux is not an issue. There are no AVC errors in 
/var/log/audit/audit.log and it fails the same way in 'Enforcing' and 
'Permissive' modes

I'm pretty certain the dogtag service is not running

Then you have your lucky winner! :-)

Some pointers to logs are for example here:
http://www.freeipa.org/page/Troubleshooting#Server_Installation


/var/log/pki-ca/catalina.out contains the lines at boot time:
INFO: Deploying web application directory ca
Nov 12, 2015 3:33:47 PM org.apache.tomcat.util.modeler.Registry 
registerComponent
SEVERE: Null component 
Catalina:type=JspMonitor,name=jsp,WebModule=//localhost/ca,J2EEApplication=none,J2EEServer=none
Nov 12, 2015 3:33:47 PM org.apache.catalina.startup.HostConfig deployDirectory
SEVERE: Error deploying web application directory ca
java.lang.UnsupportedClassVersionError: 
com/netscape/cms/servlet/filter/AgentRequestFilter : Unsupported major.minor 
version 51.0 (unable to load class 
com.netscape.cms.servlet.filter.AgentRequestFilter)
         at 
org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2334)....
 lots of traceback

/var/log/pki-ca/system is empty
/var/log/pki-ca/debug has nothing new for 2 days

CCing Fraser. This is a wild guess, but maybe you updated your java to java-1.8.0-openjdk? PKI does not work on it on RHEL/CentOS:

https://bugzilla.redhat.com/show_bug.cgi?id=1262516

java would need to be switched with "alternate" to pre-1.8.0 version if this is the case.

This fault with the "Service Unavailable" originally came up when I
tried to delete a host from the freeip gui
In the file  /var/log/dirsrv/slapd-PKI-IPA/errors file there was a Warning 
about nsslapd-cachememsize not being big enough but I don't know how to change 
it if, indeed this is anything to do with it.

This should not cause this error, it is more about performance tuning, AFAIK.
That's good to know..


The Manheim group of companies within the UK comprises: Manheim Europe Limited 
(registered number: 03183918), Manheim Auctions Limited (registered number: 
00448761), Manheim Retail Services Limited (registered number: 02838588), 
Motors.co.uk Limited (registered number: 05975777), Real Time Communications 
Limited (registered number: 04277845) and Complete Automotive Solutions Limited 
(registered number: 05302535). Each of these companies is registered in England 
and Wales with the registered office address of Central House, Leeds Road, 
Rothwell, Leeds LS26 0JE. The Manheim group of companies operates under various 
brand/trading names including Manheim Inspection Services, Manheim Auctions, 
Manheim Direct, Manheim De-fleet and Manheim Aftersales Solutions.

V:0CF72C13B2AC




--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to