On 11/12/2015 04:51 PM, Terry John wrote:
I got a core dump of certmonger failing user abrt but it's huge. Is there any
particular part that would be useful.
CCing Nalin and David for the core dump. More below.
On 11/12/2015 02:17 PM, Terry John wrote:
I had a working freeipa setup on a CentOS release 6.7 machine. All was well
until I did a yum update. Now I have multiple issue apparently based around the
CMS (Service Unavailable) issue.
My current version of ipa-server is 3.0.0-47
Certmonger crashes with a segmentation fault at boot time and crashes every
time I try to restart it when ipa is running.
It of course should not crash, it would be useful to have a backtrace from the
core file that was generated.
Here is the backtrace of the core file:
{ "signal": 11
, "executable": "/usr/sbin/certmonger"
, "stacktrace":
[ { "crash_thread": true
, "frames":
[ { "address": 140527158519285
, "build_id": "87a19a61dc011579f3e25de3ca9778c6fd9e4547"
, "build_id_offset": 1222133
, "function_name": "__strstr_sse42"
, "file_name": "/lib64/libc.so.6"
}
, { "address": 140527209363149
, "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
, "build_id_offset": 141005
, "file_name": "/usr/sbin/certmonger"
}
, { "address": 140527209301676
, "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
, "build_id_offset": 79532
, "file_name": "/usr/sbin/certmonger"
}
, { "address": 140527209287550
, "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
, "build_id_offset": 65406
, "file_name": "/usr/sbin/certmonger"
}
, { "address": 140527209291166
, "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
, "build_id_offset": 69022
, "file_name": "/usr/sbin/certmonger"
}
, { "address": 140527196303038
, "build_id": "4135efbfc51bb80e4945275a6e6ba10e9d8a2a11"
, "build_id_offset": 36542
, "file_name": "/usr/lib64/libtevent.so.0"
}
, { "address": 140527196295910
, "build_id": "4135efbfc51bb80e4945275a6e6ba10e9d8a2a11"
, "build_id_offset": 29414
, "file_name": "/usr/lib64/libtevent.so.0"
}
, { "address": 140527196279965
, "build_id": "4135efbfc51bb80e4945275a6e6ba10e9d8a2a11"
, "build_id_offset": 13469
, "function_name": "_tevent_loop_once"
, "file_name": "/usr/lib64/libtevent.so.0"
}
, { "address": 140527209278079
, "build_id": "3a90011aabc8c2612ed5fe7e1249bec8438c72b3"
, "build_id_offset": 55935
, "function_name": "main"
, "file_name": "/usr/sbin/certmonger"
} ]
} ]
}
In /var/log/messages I get
freeipasvr kernel: certmonger[2611] general protection ip:7fb487fed5f5
sp:7ffd9df46898 error:0 in libc-2.12.so[7fb487ec3000+18a000]
This is the first error I get in /var/log/httpd/error_log when I try to delete
a host
[error] ipa: ERROR: ipaserver.plugins.dogtag.ra.get_certificate(): Unable to
communicate with CMS (Service Unavailable)
If I stop ipa the start certmonger it starts ok and continues to run when I start ipa
again but as soon as any requests are made like "getcert list" then it crashes
again.
With certmonger still running I can do a request
# ipa cert-status
Request id: 20140417164153
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (Service Unavailable) # service certmonger status
certmonger (pid 3030) is running...
It looks like PKI cannot be contacted. I would recommend checking
/var/log/httpd/error_log, it may have more details. I would also recommend checking
"ipa cert-show 1", it will probably fail with the same bug.
Yes ipa cert-show 1 does show the same thing
# ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: Unable to communicate
with CMS (Service Unavailable)
Next steps may include checking that dogtag service really runs, there is no
SELinux AVC. If neither of this helps, you can check PKI logs /var/log/pki...
to see what went wrong.
I'm sure SELinux is not an issue. There are no AVC errors in
/var/log/audit/audit.log and it fails the same way in 'Enforcing' and
'Permissive' modes
I'm pretty certain the dogtag service is not running
Then you have your lucky winner! :-)
Some pointers to logs are for example here:
http://www.freeipa.org/page/Troubleshooting#Server_Installation
/var/log/pki-ca/catalina.out contains the lines at boot time:
INFO: Deploying web application directory ca
Nov 12, 2015 3:33:47 PM org.apache.tomcat.util.modeler.Registry
registerComponent
SEVERE: Null component
Catalina:type=JspMonitor,name=jsp,WebModule=//localhost/ca,J2EEApplication=none,J2EEServer=none
Nov 12, 2015 3:33:47 PM org.apache.catalina.startup.HostConfig deployDirectory
SEVERE: Error deploying web application directory ca
java.lang.UnsupportedClassVersionError:
com/netscape/cms/servlet/filter/AgentRequestFilter : Unsupported major.minor
version 51.0 (unable to load class
com.netscape.cms.servlet.filter.AgentRequestFilter)
at
org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2334)....
lots of traceback
/var/log/pki-ca/system is empty
/var/log/pki-ca/debug has nothing new for 2 days
CCing Fraser. This is a wild guess, but maybe you updated your java to
java-1.8.0-openjdk? PKI does not work on it on RHEL/CentOS:
https://bugzilla.redhat.com/show_bug.cgi?id=1262516
java would need to be switched with "alternate" to pre-1.8.0 version if this is
the case.
This fault with the "Service Unavailable" originally came up when I
tried to delete a host from the freeip gui
In the file /var/log/dirsrv/slapd-PKI-IPA/errors file there was a Warning
about nsslapd-cachememsize not being big enough but I don't know how to change
it if, indeed this is anything to do with it.
This should not cause this error, it is more about performance tuning, AFAIK.
That's good to know..
The Manheim group of companies within the UK comprises: Manheim Europe Limited
(registered number: 03183918), Manheim Auctions Limited (registered number:
00448761), Manheim Retail Services Limited (registered number: 02838588),
Motors.co.uk Limited (registered number: 05975777), Real Time Communications
Limited (registered number: 04277845) and Complete Automotive Solutions Limited
(registered number: 05302535). Each of these companies is registered in England
and Wales with the registered office address of Central House, Leeds Road,
Rothwell, Leeds LS26 0JE. The Manheim group of companies operates under various
brand/trading names including Manheim Inspection Services, Manheim Auctions,
Manheim Direct, Manheim De-fleet and Manheim Aftersales Solutions.
V:0CF72C13B2AC
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
