-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Martin, Et al,
Now that debugging is installed and running, I cannot duplicate. Isn't that always the way though? I'll let you know if it happens again. Thanks, Bill On 6/12/15 3:32 PM, Rich Megginson wrote: > On 06/12/2015 03:25 PM, William Graboyes wrote: Hi Ken, > > I ran this command back to back, I am snipping some of the > results. > > First time I ran the command: > > time ldapsearch -x -h 127.0.0.1 "(uid=admin)" # extended LDIF # # > LDAPv3 # base <dc=foo,dc=org> (default) with scope subtree # > filter: (uid=admin) # requesting: ALL # > > --snip-- > > # search result search: 2 result: 0 Success > > # numResponses: 3 # numEntries: 2 > > real 0m0.056s user 0m0.003s sys 0m0.004s > > > Run on the same server not 5 seconds after the previous command: > > time ldapsearch -x -h 127.0.0.1 "(uid=admin)" # extended LDIF # # > LDAPv3 # base <dc=foo,dc=org> (default) with scope subtree # > filter: (uid=admin) # requesting: ALL # > > -- snip -- > > # search result search: 2 result: 0 Success > > # numResponses: 3 # numEntries: 2 > > real 0m31.756s user 0m0.003s sys 0m0.005s > >> Ok. First, see >> http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes > >> You'll also have to do # debuginfo-install ipa-server slapi-nis >> to get all of the ipa packages. > >> Next, see >> http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs > >> Reproduce the problem, and during the 30 seconds the directory >> server is processing the search request, run the gdb command >> several times to get stack traces during the search request. > > > > I am starting to see this error in the dirserv logs: > > [12/Jun/2015:14:06:51 -0700] slapi_ldap_bind - Error: could not > send startTLS request: error -1 (Can't contact LDAP server) errno > 107 (Transport endpoint is not connected) [12/Jun/2015:14:11:51 > -0700] slapi_ldap_bind - Error: could not send startTLS request: > error -1 (Can't contact LDAP server) errno 107 (Transport endpoint > is not connected) [12/Jun/2015:14:16:51 -0700] slapi_ldap_bind - > Error: could not send startTLS request: error -1 (Can't contact > LDAP server) errno 107 (Transport endpoint is not connected) > [12/Jun/2015:14:21:51 -0700] slapi_ldap_bind - Error: could not > send startTLS request: error -1 (Can't contact LDAP server) errno > 107 (Transport endpoint is not connected) > >> I doubt this is related to the performance. This looks like the >> server is attempting to contact a replica which is down, and has >> backed off for the full 5 minute max backoff. > > > Thanks, Bill Graboyes > > > On 6/12/15 1:36 PM, Rich Megginson wrote: >>>> On 06/12/2015 02:10 PM, Martin Kosek wrote: >>>>> On 06/12/2015 09:15 PM, William Graboyes wrote: >>>> Hi Martin, >>>> >>>> Here are the outputs of the various commands, cleaned of >>>> course: >>>> >>>> time ldapsearch SASL/EXTERNAL authentication started >>>> ldap_sasl_interactive_bind_s: Unknown authentication method >>>> (-6) additional info: SASL(-4): no mechanism available: >>>> >>>> real 0m32.464s user 0m0.385s sys 0m0.052s >>>>>> This is quite long time. We should check respective >>>>>> dirsrv errors and access logs snippets. >>>>>> >>>>>> Also, the command above did not exit successfully, I >>>>>> would recommend doing at least >>>>>> >>>>>> # ldapsearch -x -h `hostname` "(uid=admin)" >>>>> To eliminate DNS from the equation, use # time ldapsearch >>>>> -x -h 127.0.0.1 "(uid=admin)" >>>> time host ipa-server-2.foo.org <-- server with issues >>>> ipa-server-2.foo.org has address 10.0.0.2 >>>> >>>> real 0m0.070s user 0m0.010s sys 0m0.006s >>>> >>>> time host ipa-server-1.foo.org <-- replicant with no issues >>>> ipa-server-1.foo.org has address 10.0.0.3 >>>> >>>> real 0m0.073s user 0m0.012s sys 0m0.006s >>>> >>>> time kinit kinit: Cannot contact any KDC for realm 'FOO.ORG' >>>> while getting initial credentials >>>> >>>> real 0m27.049s user 0m0.013s sys 0m0.004s >>>> >>>> ^^^ has been something I have been seeing intermittently >>>> >>>> >>>> >>>> On 6/12/15 12:11 AM, Martin Kosek wrote: >>>>>>>>> Hi List, >>>>>>>>> >>>>>>>>> This is a problem that has surfaced after a reboot >>>>>>>>> of this system in particular. It is being really, >>>>>>>>> really slow. In terms of hardware usage issues, >>>>>>>>> there are none. It is taking 3-5 minutes to list >>>>>>>>> users in the gui. Running commands like >>>>>>>>> ipa-replica-manage list is taking between 30seconds >>>>>>>>> and 3 minutes. Memory usage is low, cpu usage is >>>>>>>>> low, iops are low. I really have no idea where to >>>>>>>>> start here, there is noting really damning in the >>>>>>>>> logs. I have tried restarting IPA (ipactl >>>>>>>>> restart) stopping and starting IPA (ipactl stop >>>>>>>>> wait... ipactl start), and rebooting the entire >>>>>>>>> server. >>>>>>>>> >>>>>>>>> The oddest thing is that there have been some krb >>>>>>>>> errors saying that they cannot contact the krb >>>>>>>>> server.. logging into the gui saying your session >>>>>>>>> has timed out.. >>>>>>>>> >>>>>>>>> It is just general strangeness. >>>>>>>>> >>>>>>>>> ipa-server-4.1.0-18.el7.centos.3.x86_64 >>>>>>>>> sssd-ipa-1.12.2-58.el7_1.6.x86_64 >>>>>>>>> krb5-server-1.12.2-14.el7.x86_64 >>>>>>>>> >>>>>>>>> Any help would be greatly appreciated. >>>>>>>>> >>>>>>>>> Thanks, Bill >>>>>>>> I would recommend starting with simple things, seeing >>>>>>>> the performance and then following with more complex >>>>>>>> stuff: >>>>>>>> >>>>>>>> - Try bare "ldapsearch" against the FreeIPA LDAP >>>>>>>> server, see the response rate. If it is also slow, we >>>>>>>> have the root cause. Before ringing on DS people >>>>>>>> doors, see if for example DNS is not slow and there >>>>>>>> are no DNS timeouts in play - "host ipa.server.test" >>>>>>>> will tell you that >>>>>>>> >>>>>>>> - If DS is OK, try Kerberos - kinit, kvno commands >>>>>>>> >>>>>>>> - If Kerberos is also OK and "ipa-replica-manage >>>>>>>> list" is still slow, maybe we should just "strace" it >>>>>>>> to see what it waits on. >>>>>>>> >>>>>>>> HTH, Martin >>>>>>>> >> > -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJVe2ZIAAoJEJFMz73A1+zr8qgP/jmKVGOwFMVZFowdB8rPBHOT //TQ11V5E0nhMz3r0zWoUDOEMYyQy5Oy9d4CmoMa5oWimnTHJUJanUMK5hWHlo/g d8lSyvEDmQbXpNCduwIeMwupI0C669X5/EYesvth3khebsHxlz7kov69XyknBcj2 8czY4eUAkS8jB60Ua4UOMa0ruuBtYU40FCdS8GiWFBYxjfnLLqUX4vsfyke+vJ0B AYPlzsf0ipKdPVPhjxaWmjHJmU/Y0tK5/a8CrDpkQH19UzYjFX8BSpyAJBGOQVYw 4ZlhZHXmiGhuDnyoIZIHFOeo0BmGugiN85zLf4G4mFxkn2TNOp28+w94EBZxb9kI rQ1vEE2eUF0f9n5usdXb+gHwm3yhnhOvOkV+MLhJXNTeTlEo9Kl/EEnWZamh4wRy hsMP2j6/XeDDzNFd4q1JaiScGVwfzIAizFGzxz6jkluA8B/aCz05pjMVDf/HmLPh 64OygyzhYtkLTe6DTH/WwoLV664IDlzs6LMxEDix37dI+9e8TsLfdp9ZexQV24sR qZEYqWcPqDfXPvjkXjsqmeU7mJMOaQsu7be1Ad9isoOocn1WorIx7eCFtHIT5xgF D58uhXk6hHj1tYA3fDQY2ybWgD0HhST52XbRezwQQ2Mw4F5QcUBt/WtqEPwMVlDU Pfp6LpG4V0Kph4zc3EF3 =IOQh -----END PGP SIGNATURE----- -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
