Hi Daryl,
Thanks for all the data. I will look at the pstacks. A first look shows
that you capture import, bind... so may be a complete
ipa-replica-install session.
I will try to retrieve the specific startup time to see what was going
on at that time.
If you have the time to monitor only startup, it will help me shrinking
the set of pstacks.
Startup of DS last > 1min. If you may start DS and as soon as the
ns-slapd process is launched, do regular pstacks. Then when you are able
to send a simple ldapsearch (ldapsearch -x -b "" -s base), you may stop
taking pstacks.
thanks
thierry
On 03/14/2016 03:06 PM, Daryl Fonseca-Holt wrote:
Hi Thierry,
I moved the old logs into a subdirectory called try1. I did the
recommended ipa-server-install --uninstall. Tried the replica install
again. Failed during kadmind start like the previous time.
The log from ipa-replica-install (with -d) is at
http://home.cc.umanitoba.ca/~fonsecah/ipa/ipareplica-install.log
The console script (mostly the same as the log but with my entries) is
at http://home.cc.umanitoba.ca/~fonsecah/ipa/ipa-replica-install.console
The 5 second pstacks are at
http://home.cc.umanitoba.ca/~fonsecah/ipa/slapd-pstacks.console
Thanks, Daryl
On 03/11/16 02:40, thierry bordaz wrote:
Hello Deryl,
My understanding is that ns-slapd is first slow to startup. Then
when krb5kdc is starting it may load ns-slapd.
We identified krb5kdc may be impacted by the number of users
accounts.
From the ns-slapd errors log it is not clear why it is so slow to
start.
Would you provide the ns-slapd access logs from that period.
Also in order to know where ns-slapd is spending time, it would
really help if you can get regular (each 5s) pstacks (with
389-ds-debuginfo), during DS startup and then later during
krb5kdc startup.
best regards
thierry
On 03/10/2016 11:10 PM, Daryl Fonseca-Holt wrote:
Environment:
RHEL 7.2
IPA 4.2.0-15
nss 3.19.1-19
389-ds-base 1.3.4.0-26
sssd 1.13.0-40
I've encountered this problem in IPA 3.0.0 but hoped it was
addressed in 4.2.0.
Trying to set up a replica of a master with 150,000+ user accounts,
NIS and Schema Compatability enabled on the master.
During ipa-replica-install it attempts to start IPA. dirsrv starts,
krb5kdc starts, but then kadmind fails because krb5kdc has gone
missing.
This happens during restart of IPA in version 3.0.0 too. There it
can be overcome by manually starting each component of IPA _but_
waiting until ns-slapd-<instance> has settled down (as seen from
top) before starting krb5kdc. I also think that the startup of
krb5kdc loads the LDAP instance quite a bit.
There is a problem in the startup logic where dirsrv is so busy that
even though krb5kdc successfully starts and allows the kadmin to
begin kdb5kdc is not really able to do its duties.
I'm reporting this since there must be some way to delay the start
of krb5kdc and then kadmind until ns-slapd-<instance> is really open
for business.
# systemctl status krb5kdc.service
● krb5kdc.service - Kerberos 5 KDC
Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service;
disabled; vendor preset: disabled)
Active: inactive (dead)
Mar 10 14:19:13 jutta.cc.umanitoba.ca systemd[1]: Stopped Kerberos 5
KDC.
Mar 10 14:20:36 jutta.cc.umanitoba.ca systemd[1]: Starting Kerberos
5 KDC...
Mar 10 14:20:39 jutta.cc.umanitoba.ca systemd[1]: Started Kerberos 5
KDC.
# systemctl status krb5kdc.service
● krb5kdc.service - Kerberos 5 KDC
Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service;
disabled; vendor preset: disabled)
Active: inactive (dead)
Mar 10 14:19:13 jutta.cc.umanitoba.ca systemd[1]: Stopped Kerberos 5
KDC.
Mar 10 14:20:36 jutta.cc.umanitoba.ca systemd[1]: Starting Kerberos
5 KDC...
Mar 10 14:20:39 jutta.cc.umanitoba.ca systemd[1]: Started Kerberos 5
KDC.
journalctl -xe was stale by the time I got to it so I've attached
/var/log/messages instead.
The log from ipa-replica-install (with -d) is at
http://home.cc.umanitoba.ca/~fonsecah/ipa/ipareplica-install.log
The console script (mostly the same as the log but with my entries)
is at
http://home.cc.umanitoba.ca/~fonsecah/ipa/ipa-replica-install.console
The /var/log/dirsrv/ns-slapd-<instance> access log is at
http://home.cc.umanitoba.ca/~fonsecah/ipa/access
Regards, Daryl
--
--
Daryl Fonseca-Holt
IST/CNS/Unix Server Team
University of Manitoba
204.480.1079
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
