On 11/13/2014 03:02 AM, Walter van Lille wrote:
Thanks Rich, I have installed the packages and run gdb again. Hopefully the attached file is more useful.

The symbols are there. However, the server is almost completely idle - no hangs, no deadlocks, no waiting on I/O.

You must catch dirsrv while it is hung. You might want to run that gdb script every few seconds during the time it appears to be hung.


On Wed, Nov 12, 2014 at 4:16 PM, Rich Megginson <rmegg...@redhat.com <mailto:rmegg...@redhat.com>> wrote:

    On 11/12/2014 05:42 AM, Walter van Lille wrote:
    Thanks again for the assistance guys. I have saved two files and
    included it here. Hope it tells you more than it does me.

    These stack traces contain no useful symbols.

    Please read
    http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes
    to find out how to install the correct debuginfo packages on your
    system.  For IPA you will also need debuginfo packages for ipa and
    slapi-nis

    If debuginfo-install is not working, see
    https://www.centos.org/forums/viewtopic.php?t=1710

    If you simply cannot figure out how to install the debuginfo
    packages, please email me with the following information:

    $ rpm -q ipa-server slapi-nis 389-ds-base openldap db4 nss nspr glibc

    and I will make them available to you

    NOTE: With debuginfo packages, the version of the debuginfo
    package _must exactly match_ the version of the associated package
    e.g. for 389-ds-base.x86_64 1.2.11.15-34.el6_5 you must have
    389-ds-base-debuginfo.x86_64   1.2.11.15-34.el6_5

    If the versions do not match exactly, you will not get the symbols.


    Regards,

    Wallter

    On Tue, Nov 11, 2014 at 8:03 PM, Rich Megginson
    <rmegg...@redhat.com <mailto:rmegg...@redhat.com>> wrote:

        On 11/11/2014 10:37 AM, Martin Basti wrote:
        On 11/11/14 15:58, Rich Megginson wrote:
        On 11/11/2014 06:20 AM, Ludwig Krispenz wrote:

        On 11/11/2014 02:14 PM, Martin Basti wrote:
        Ludiwg (CCed) this seems like old (fixed?) DS bug.
        hmm, it says limit is 2097152, so it already has the new
        setting, but the error message says the packet is 800MB*
        *

        *Right.  That usually means the server was expecting an
        encrypted SASL buffer from the client, but instead the
        client thinks SASL encryption negotiation failed and just
        sent a plain LDAP buffer.  What version of 389-ds-base are
        you using?  rpm -q 389-ds-base

        https://fedorahosted.org/389/ticket/47416

        So, DO NOT increase your sasl io buffer size - it will not
        fix the problem, and it will leave you open to DoS attacks.
        *

        He is using

        CentOS release 6.5 (Final)
        389-ds-base.x86_64 1.2.11.15-34.el6_5


        Ignore the "SASL encrypted packet length exceeds maximum
        allowed limit" error.  All it means is the client has some
        error and is canceling the connection.

        The bugzilla associated with *47416
        <https://fedorahosted.org/389/ticket/47416> is targeted for
        RHEL 7.1.  The problem is that we were never able to figure
        out what error the client was getting that caused the client
        to stop establishing the *SASL encryption, and the original
        customer who reported the problem dropped the case -
        everything just started working, no further errors.  Note
        that 47416 doesn't really fix the problem or address the root
        cause - the root cause is some error on the client side that
        causes it to stop doing encrypted SASL I/O and send an LDAP
        unbind request.

        Let's get back to the actual problem - what is the actual
        problem?  The ldap server is hanging or unresponsive?  If so,
        then see
        http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs.
        Let's get some dirsrv stack traces during the period of time
        when it appears to be unresponsive.



        *
        *
        **

        On 11/11/14 13:13, Walter van Lille wrote:
        I've just cleaned out a ton of slapd_poll timed out
        messages from the output and changed the names to
        protect the innocent, :-)
        Here is the output as requested:


        *[05/Nov/2014:11:44:05 +0200] - SASL encrypted packet
        length exceeds maximum allowed limit (length=805565,
        limit=2097152). Change the nsslapd-maxsasliosize
        attribute in cn=config to increase limit.*
        *
        *
        *[10/Nov/2014:14:45:19 +0200] - slapd_poll(115) timed out*
        *[10/Nov/2014:14:45:19 +0200] sasl_io_enable - Cannot
        enable SASL security on connection in CLOSING state*
        *[10/Nov/2014:14:45:19 +0200] - Error: could not
        add/remove IO layers from connection*
        *[11/Nov/2014:11:48:09 +0200] - slapd shutting down -
        signaling operation threads*
        *[11/Nov/2014:11:48:09 +0200] - slapd shutting down -
        waiting for 30 threads to terminate*
        *[11/Nov/2014:13:14:12 +0200] - slapd shutting down -
        closing down internal subsystems and plugins*
        *[11/Nov/2014:13:14:12 +0200] - Waiting for 4 database
        threads to stop*
        *[11/Nov/2014:13:14:13 +0200] - All database threads now
        stopped*
        *[11/Nov/2014:13:14:13 +0200] - slapd stopped.*
        *[11/Nov/2014:13:26:35 +0200] - 389-Directory/1.2.11.15
        <http://1.2.11.15> B2014.219.179 starting up*
        *[11/Nov/2014:13:26:35 +0200] schema-compat-plugin -
        warning: no entries set up under cn=computers,
        cn=compat,dc=sample,dc=example*
        *[11/Nov/2014:13:26:36 +0200] - Skipping CoS Definition
        cn=Password Policy,cn=accounts,dc=sample,dc=example--no
        CoS Templates found, which should be added before the
        CoS Definition.*
        *[11/Nov/2014:13:26:36 +0200] - Skipping CoS Definition
        cn=Password Policy,cn=accounts,dc=sample,dc=example--no
        CoS Templates found, which should be added before the
        CoS Definition.*
*[11/Nov/2014:13:26:36 +0200] - slapd started. Listening on All Interfaces port 389 for LDAP requests*
        *[11/Nov/2014:13:26:36 +0200] - Listening on All
        Interfaces port 636 for LDAPS requests*
        *[11/Nov/2014:13:26:36 +0200] - Listening on
        /var/run/slapd-SAMPLE-EXAMPLE.socket for LDAPI requests*
        *[11/Nov/2014:13:57:08 +0200] - slapd_poll(78) timed out*
        *
        *
        *
        *
        *
        *


        On Tue, Nov 11, 2014 at 1:19 PM, Martin Basti
        <mba...@redhat.com <mailto:mba...@redhat.com>> wrote:

            IMHO It's DS bug, can you share DS error log?
            pspacek CCed to examine named logs.

            Martin^2


            On 11/11/14 12:13, Walter van Lille wrote:
            Hi Martin, thanks for the reply.
            My version: bind-dyndb-ldap-2.3-5.el6.x86_64
            The server doesn't have journalctl installed but I
            have the outputs from the messages and named.run
            files that I included here:

            Messages:

            *Nov 11 12:30:13 freeipa named[1481]: error
            (network unreachable) resolving
            'example.example.com.10.123.123.123/A/IN':
            2001:500:2f::f#53*
            *Nov 11 12:30:23 freeipa named[1481]: LDAP query
            timed out. Try to adjust "timeout" parameter*
            *Nov 11 12:30:23 freeipa named[1481]: LDAP query
            timed out. Try to adjust "timeout" parameter*
            *Nov 11 12:30:33 freeipa named[1481]: LDAP query
            timed out. Try to adjust "timeout" parameter*
            *Nov 11 12:30:33 freeipa named[1481]: LDAP query
            timed out. Try to adjust "timeout" parameter*

            Named.run:

            *client 10.123.123.123#42639: transfer of
            'example.example/IN': AXFR-style IXFR started*
            *client 10.123.123.123#42639: transfer of
            ''example.example/IN': AXFR-style IXFR ended*
            *client 10.123.123.123#46912: transfer of
            '10.123.123.123.in-addr.arpa/IN': AXFR-style IXFR
            started*
            *client 10.123.123.123#46912: transfer of
            '10.123.123.123.in-addr.arpa/IN': AXFR-style IXFR
            ended*
            *LDAP query timed out. Try to adjust "timeout"
            parameter*
            *LDAP query timed out. Try to adjust "timeout"
            parameter*
            *LDAP query timed out. Try to adjust "timeout"
            parameter*

            I just replaced the IPs and the actual names with
            something more generic.

            Regards,

            Walter

            On Thu, Nov 6, 2014 at 5:00 PM, Martin Basti
            <mba...@redhat.com <mailto:mba...@redhat.com>> wrote:

                On 06/11/14 14:58, Walter van Lille wrote:
                Hi,

                I need some assistance please.
                I've taken over an IPA server to manage a few
                months ago, and it was working fine until
                recently when it started acting up seemingly
                off its own accord.
                When I do an ipactl status it basically gives
                an output as shown below:


                *Directory Service: RUNNING
                *
                *
                *
                *Loooooooooooooooooooooooooooooooooooooooooooooooooong
                pause... (To the tune of 7 minutes sometimes)*
                *
                *
                *KDC Service: RUNNING*
                *KPASSWD Service: RUNNING*
                *DNS Service: RUNNING*
                *MEMCACHE Service: RUNNING*
                *HTTP Service: RUNNING*
                *CA Service: RUNNING*
                *ADTRUST Service: RUNNING*
                *EXTID Service: RUNNING*

                Running top showed that ns-slapd was munching
                almost all my resources, but I got that fixed
                by upping the cache. Unfortunately this did
                not correct the issue and it still reacts in
                the same fashion, although the resources have
                been freed up now.
                I've noticed that when I run dig on either the
                local server or a remote machine that the
                query basically just times out as shown here:

                *dig freeipa.myexample.sample*
                *
                *
                *; <<>> DiG
                9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>>
                freeipa.myexample.sample*
                *;; global options: +cmd*
                *;; connection timed out; no servers could be
                reached*

                When the KDC service fails to start, then name
                lookups seem OK, but authentication fails.
                otherwise it's dead in the water.

                This also happens:

                *sudo ipactl status*
                *Directory Service: RUNNING*
                *Unknown error when retrieving list of
                services from LDAP:*
                *
                *
                My software setup is as follows:

                *CentOS release 6.5 (Final)
                *
                *389-ds-base.x86_64 1.2.11.15-34.el6_5
                *
                *bind.x86_64  32:9.8.2-0.23.rc1.el6_5.1
                *
                *bind-dyndb-ldap.x86_64*
                *bind-libs.x86_64 32:9.8.2-0.23.rc1.el6_5.1*
                *bind-utils.x86_64  32:9.8.2-0.23.rc1.el6_5.1*
                *rpcbind.x86_64 0.2.0-11.el6
                @anaconda-CentOS-201311291202.x86_64/6.5*
                *samba4-winbind.x86_64*
                *krb5-server.x86_64 1.10.3-15.el6_5.1
                *
                *
                *
                *Linux 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue
                Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64
                GNU/Linux
                *

                It's not a permanent situation as it sometimes
                runs 100% for a while, but 80% of the time it
                is unusable. If anybody can assist me, please
                be so kind.

                Regards,

                Walter

                Hello please which version of bind-dyndb-ldap
                do you use?
                I had similar issue with bind-dyndb-ldap, but
                it was development version, I'm not sure if
                this is your case.
                When named was failing, dirserv was really slow.

                Can you send journalctl -b -u named log when
                dig doesn't work??

-- Martin Basti




-- Martin Basti




-- Martin Basti








-- Martin Basti




        --
        Manage your subscription for the Freeipa-users mailing list:
        https://www.redhat.com/mailman/listinfo/freeipa-users
        Go To http://freeipa.org for more info on the project





-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

Reply via email to