Re: [Freeipa-users] Server randomly will stop accepting krb requests
On Mon, 30 Sep 2013, Andrew Tranquada wrote:
I have 6 servers setup as freeipa replicas.
5 are working great, no problems.
They are all running ipa-server-3.0.0-26.el6_4.4.x86_64
However, the same one will randomly stop working. By stop working I mean the
following:
(domain name and ips have been redacted)
I cannot kinit as any user on that machine:
[root@badserver ~]# kinit admin
kinit: Generic error (see e-text) while getting initial credentials
I cannot connect on 389 or 636 to that server:
telnet badserver 636
telnet: Unable to connect to remote host: Connection refused
slapd is running and listening on port 389 according to netstat:
[root@badserver ~]# netstat -lpn | grep 389
tcp0 0 :::7389 :::*
LISTEN 16419/ns-slapd
This is port 7389, for CA LDAP instance, not port 389 which is main LDAP
instance.
but nothing is returned for port 636
Because port 636 is served by the same main dirsrv instance that is
down.
in the /var/log/slapd-PKI* or slapd-DOMAIN error files, the last error is
from over a week ago, actually the last entry period is from there.
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
/var/log/krb5kdc.log shows
Sep 30 12:22:24 badserver krb5kdc[32063](info): AS_REQ (4 etypes {18 17 16 23})
ip: LOOKING_UP_CLIENT: ad...@example.com for krbtgt/example@example.com,
Server error
a service ipa restart ALWAYS fixes it.
Directory server instance is down, so LDAP server is not accessible, so
Kerberos KDC cannot read the data which is only in LDAP, so it denies
access.
Any guidance/advice/docs to read would be greatly appreciated! The fact
that it seems to be so random and the other 5 ipa servers are working
great makes it even more frustrating!
Look at directory server's logs to see what was the reason for refusing
starting up in /var/log/dirsrv/slapd-DOMAIN/errors.
--
/ Alexander Bokovoy
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Server randomly will stop accepting krb requests
Thanks for the response
I did look in /var/log/slapd-PKI* or slapd-DOMAIN (I guess I was not too
clear I did that in my email)
in those logs the last thing in that log is from Sep 18
From /var/log/dirsrv/slapd-EXAMPLE-COM/errors:
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
That is all, the items before that time are addition/deletion of entries which
is normal.
-Original Message-
From: Alexander Bokovoy aboko...@redhat.com
Sent: Monday, September 30, 2013 12:47pm
To: Andrew Tranquada andrew.tranqu...@rackspace.com
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] Server randomly will stop accepting krb requests
On Mon, 30 Sep 2013, Andrew Tranquada wrote:
I have 6 servers setup as freeipa replicas.
5 are working great, no problems.
They are all running ipa-server-3.0.0-26.el6_4.4.x86_64
However, the same one will randomly stop working. By stop working I mean the
following:
(domain name and ips have been redacted)
I cannot kinit as any user on that machine:
[root@badserver ~]# kinit admin
kinit: Generic error (see e-text) while getting initial credentials
I cannot connect on 389 or 636 to that server:
telnet badserver 636
telnet: Unable to connect to remote host: Connection refused
slapd is running and listening on port 389 according to netstat:
[root@badserver ~]# netstat -lpn | grep 389
tcp0 0 :::7389 :::*
LISTEN 16419/ns-slapd
This is port 7389, for CA LDAP instance, not port 389 which is main LDAP
instance.
but nothing is returned for port 636
Because port 636 is served by the same main dirsrv instance that is
down.
in the /var/log/slapd-PKI* or slapd-DOMAIN error files, the last error is
from over a week ago, actually the last entry period is from there.
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
/var/log/krb5kdc.log shows
Sep 30 12:22:24 badserver krb5kdc[32063](info): AS_REQ (4 etypes {18 17 16
23}) ip: LOOKING_UP_CLIENT: ad...@example.com for
krbtgt/example@example.com, Server error
a service ipa restart ALWAYS fixes it.
Directory server instance is down, so LDAP server is not accessible, so
Kerberos KDC cannot read the data which is only in LDAP, so it denies
access.
Any guidance/advice/docs to read would be greatly appreciated! The fact
that it seems to be so random and the other 5 ipa servers are working
great makes it even more frustrating!
Look at directory server's logs to see what was the reason for refusing
starting up in /var/log/dirsrv/slapd-DOMAIN/errors.
--
/ Alexander Bokovoy
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Server randomly will stop accepting krb requests
Andrew Tranquada wrote:
Thanks for the response
I did look in /var/log/slapd-PKI* or slapd-DOMAIN (I guess I was not too
clear I did that in my email)
in those logs the last thing in that log is from Sep 18
From /var/log/dirsrv/slapd-EXAMPLE-COM/errors:
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
That is all, the items before that time are addition/deletion of entries which
is normal.
-Original Message-
From: Alexander Bokovoy aboko...@redhat.com
Sent: Monday, September 30, 2013 12:47pm
To: Andrew Tranquada andrew.tranqu...@rackspace.com
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] Server randomly will stop accepting krb requests
On Mon, 30 Sep 2013, Andrew Tranquada wrote:
I have 6 servers setup as freeipa replicas.
5 are working great, no problems.
They are all running ipa-server-3.0.0-26.el6_4.4.x86_64
However, the same one will randomly stop working. By stop working I mean the
following:
(domain name and ips have been redacted)
I cannot kinit as any user on that machine:
[root@badserver ~]# kinit admin
kinit: Generic error (see e-text) while getting initial credentials
I cannot connect on 389 or 636 to that server:
telnet badserver 636
telnet: Unable to connect to remote host: Connection refused
slapd is running and listening on port 389 according to netstat:
[root@badserver ~]# netstat -lpn | grep 389
tcp0 0 :::7389 :::*
LISTEN 16419/ns-slapd
This is port 7389, for CA LDAP instance, not port 389 which is main LDAP
instance.
but nothing is returned for port 636
Because port 636 is served by the same main dirsrv instance that is
down.
in the /var/log/slapd-PKI* or slapd-DOMAIN error files, the last error is
from over a week ago, actually the last entry period is from there.
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
/var/log/krb5kdc.log shows
Sep 30 12:22:24 badserver krb5kdc[32063](info): AS_REQ (4 etypes {18 17 16 23})
ip: LOOKING_UP_CLIENT: ad...@example.com for krbtgt/example@example.com,
Server error
a service ipa restart ALWAYS fixes it.
Directory server instance is down, so LDAP server is not accessible, so
Kerberos KDC cannot read the data which is only in LDAP, so it denies
access.
Any guidance/advice/docs to read would be greatly appreciated! The fact
that it seems to be so random and the other 5 ipa servers are working
great makes it even more frustrating!
Look at directory server's logs to see what was the reason for refusing
starting up in /var/log/dirsrv/slapd-DOMAIN/errors.
I'd look for evidence in /var/log/messages of ns-slapd core dumping.
rob
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Server randomly will stop accepting krb requests
On Mon, 30 Sep 2013, Andrew Tranquada wrote: Thanks for the response I did look in /var/log/slapd-PKI* or slapd-DOMAIN (I guess I was not too clear I did that in my email) in those logs the last thing in that log is from Sep 18 From /var/log/dirsrv/slapd-EXAMPLE-COM/errors: [18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (KDC returned error string: PROCESS_TGS)) errno 2 (No such file or directory) That is all, the items before that time are addition/deletion of entries which is normal. 'PROCESS_TGS' error message most likely means that ns-slapd failed to serve a query from KDC's database driver and disappeared, thus breaking unix domain socket that the driver was using to communicate with ns-slapd (we see it with 'errno 2 (No such file or directory)' error message). As Rob said, there should be ns-slapd core somewhere that should tell where it crashed. -- / Alexander Bokovoy ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Server randomly will stop accepting krb requests
Well I feel silly for not checking this earlier. You were correct.
Sep 18 01:09:35 freeipa1 kernel: : ns-slapd[16553]: segfault at 4 ip
0041227a sp 7fb9d15edc68 error 4 in ns-slapd[40+53000]
I am installing the 389-ds-base-debuginfo and accompanying packages now,
restarting ipa, enabling core dumps in the kernel and changing core file size
to unlimited.
Will see what happens next! Thanks!
-Original Message-
From: Rob Crittenden rcrit...@redhat.com
Sent: Monday, September 30, 2013 1:13pm
To: Andrew Tranquada andrew.tranqu...@rackspace.com, Alexander Bokovoy
aboko...@redhat.com
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] Server randomly will stop accepting krb requests
Andrew Tranquada wrote:
Thanks for the response
I did look in /var/log/slapd-PKI* or slapd-DOMAIN (I guess I was not too
clear I did that in my email)
in those logs the last thing in that log is from Sep 18
From /var/log/dirsrv/slapd-EXAMPLE-COM/errors:
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
That is all, the items before that time are addition/deletion of entries
which is normal.
-Original Message-
From: Alexander Bokovoy aboko...@redhat.com
Sent: Monday, September 30, 2013 12:47pm
To: Andrew Tranquada andrew.tranqu...@rackspace.com
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] Server randomly will stop accepting krb requests
On Mon, 30 Sep 2013, Andrew Tranquada wrote:
I have 6 servers setup as freeipa replicas.
5 are working great, no problems.
They are all running ipa-server-3.0.0-26.el6_4.4.x86_64
However, the same one will randomly stop working. By stop working I mean the
following:
(domain name and ips have been redacted)
I cannot kinit as any user on that machine:
[root@badserver ~]# kinit admin
kinit: Generic error (see e-text) while getting initial credentials
I cannot connect on 389 or 636 to that server:
telnet badserver 636
telnet: Unable to connect to remote host: Connection refused
slapd is running and listening on port 389 according to netstat:
[root@badserver ~]# netstat -lpn | grep 389
tcp0 0 :::7389 :::*
LISTEN 16419/ns-slapd
This is port 7389, for CA LDAP instance, not port 389 which is main LDAP
instance.
but nothing is returned for port 636
Because port 636 is served by the same main dirsrv instance that is
down.
in the /var/log/slapd-PKI* or slapd-DOMAIN error files, the last error is
from over a week ago, actually the last entry period is from there.
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
/var/log/krb5kdc.log shows
Sep 30 12:22:24 badserver krb5kdc[32063](info): AS_REQ (4 etypes {18 17 16
23}) ip: LOOKING_UP_CLIENT: ad...@example.com for
krbtgt/example@example.com, Server error
a service ipa restart ALWAYS fixes it.
Directory server instance is down, so LDAP server is not accessible, so
Kerberos KDC cannot read the data which is only in LDAP, so it denies
access.
Any guidance/advice/docs to read would be greatly appreciated! The fact
that it seems to be so random and the other 5 ipa servers are working
great makes it even more frustrating!
Look at directory server's logs to see what was the reason for refusing
starting up in /var/log/dirsrv/slapd-DOMAIN/errors.
I'd look for evidence in /var/log/messages of ns-slapd core dumping.
rob
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Server randomly will stop accepting krb requests
On 09/30/2013 11:27 AM, Andrew Tranquada wrote:
Well I feel silly for not checking this earlier. You were correct.
Sep 18 01:09:35 freeipa1 kernel: : ns-slapd[16553]: segfault at 4 ip
0041227a sp 7fb9d15edc68 error 4 in ns-slapd[40+53000]
I am installing the 389-ds-base-debuginfo and accompanying packages now,
restarting ipa, enabling core dumps in the kernel and changing core file size
to unlimited.
http://port389.org/wiki/FAQ#Debugging_Crashes
Will see what happens next! Thanks!
-Original Message-
From: Rob Crittenden rcrit...@redhat.com
Sent: Monday, September 30, 2013 1:13pm
To: Andrew Tranquada andrew.tranqu...@rackspace.com, Alexander Bokovoy
aboko...@redhat.com
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] Server randomly will stop accepting krb requests
Andrew Tranquada wrote:
Thanks for the response
I did look in /var/log/slapd-PKI* or slapd-DOMAIN (I guess I was not too
clear I did that in my email)
in those logs the last thing in that log is from Sep 18
From /var/log/dirsrv/slapd-EXAMPLE-COM/errors:
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
That is all, the items before that time are addition/deletion of entries which
is normal.
-Original Message-
From: Alexander Bokovoy aboko...@redhat.com
Sent: Monday, September 30, 2013 12:47pm
To: Andrew Tranquada andrew.tranqu...@rackspace.com
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] Server randomly will stop accepting krb requests
On Mon, 30 Sep 2013, Andrew Tranquada wrote:
I have 6 servers setup as freeipa replicas.
5 are working great, no problems.
They are all running ipa-server-3.0.0-26.el6_4.4.x86_64
However, the same one will randomly stop working. By stop working I mean the
following:
(domain name and ips have been redacted)
I cannot kinit as any user on that machine:
[root@badserver ~]# kinit admin
kinit: Generic error (see e-text) while getting initial credentials
I cannot connect on 389 or 636 to that server:
telnet badserver 636
telnet: Unable to connect to remote host: Connection refused
slapd is running and listening on port 389 according to netstat:
[root@badserver ~]# netstat -lpn | grep 389
tcp0 0 :::7389 :::*
LISTEN 16419/ns-slapd
This is port 7389, for CA LDAP instance, not port 389 which is main LDAP
instance.
but nothing is returned for port 636
Because port 636 is served by the same main dirsrv instance that is
down.
in the /var/log/slapd-PKI* or slapd-DOMAIN error files, the last error is
from over a week ago, actually the last entry period is from there.
[18/Sep/2013:01:09:34 -0400] slapd_ldap_sasl_interactive_bind - Error: could
not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2 (Local
error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
Minor code may provide more information (KDC returned error string:
PROCESS_TGS)) errno 2 (No such file or directory)
/var/log/krb5kdc.log shows
Sep 30 12:22:24 badserver krb5kdc[32063](info): AS_REQ (4 etypes {18 17 16 23})
ip: LOOKING_UP_CLIENT: ad...@example.com for krbtgt/example@example.com,
Server error
a service ipa restart ALWAYS fixes it.
Directory server instance is down, so LDAP server is not accessible, so
Kerberos KDC cannot read the data which is only in LDAP, so it denies
access.
Any guidance/advice/docs to read would be greatly appreciated! The fact
that it seems to be so random and the other 5 ipa servers are working
great makes it even more frustrating!
Look at directory server's logs to see what was the reason for refusing
starting up in /var/log/dirsrv/slapd-DOMAIN/errors.
I'd look for evidence in /var/log/messages of ns-slapd core dumping.
rob
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
