Thanks Bret. Regarding the LDAP understanding, I feel likewise. Thankfully there are some gods out there wishing to help mere mortals like us :)
Cheers — Givaldo Lins > On Feb 15, 2018, at 11:06 AM, Bret Wortman <bret.wort...@damascusgrp.com> > wrote: > > Welcome to the thread! Happy to share the pain. > > I really wish I understood LDAP more than I do. I hope our eventual answer > solves both our problems. > > Cheers, > > > Bret > > >> On 02/15/2018 01:15 PM, Givaldo Lins via FreeIPA-users wrote: >> Hi Flo >> >> Sorry for jumping into the thread like this, but I have been following this >> because I am facing the same issue, and in my case your train of thought >> makes completly sense. The only difference is that I used my account when >> searching the LDAP tree, except by the last one that I used the admin >> account. >> I got these results: >> >> GSS-SPNEGO - numEntries: 5000 >> GSSAPI - numEntries: 5173 >> admin account: numEntries: 5000 >> >> Also, when checking the access log, I noticed that after the BIND the result >> returns my user and not the anonymous-limit. >> >> My question is: What limit is being applied to me and where could I increase >> it? >> >> Thanks, >> >> Givaldo Lins | Linux Systems Administrator >> GPG Fingerprint: A81A 14CC FA18 4273 9CC6 8945 BEDA 981C 9C4E 388A >> Canada: +1 (604) 366-5482 >> Brasil: +55 (81) 98205-1735 >> skype: givaldolins >> >> >> LinkedIn | Email >> >> ----- Mensagem original ----- >> De: "Florence Blanc-Renaud via FreeIPA-users" >> <freeipa-users@lists.fedorahosted.org> >> Para: "FreeIPA users list" <freeipa-users@lists.fedorahosted.org> >> Cc: "Bret Wortman" <bret.wort...@damascusgrp.com>, "Florence Blanc-Renaud" >> <f...@redhat.com> >> Enviadas: Quinta-feira, 15 de fevereiro de 2018 9:27:48 >> Assunto: [Freeipa-users] Re: Fixing limit on DNS searches >> >>> On 02/15/2018 05:01 PM, Bret Wortman via FreeIPA-users wrote: >>>> On 02/15/2018 09:29 AM, Florence Blanc-Renaud wrote: >>>>> On 02/15/2018 02:40 PM, Bret Wortman via FreeIPA-users wrote: >>>>>> On 02/15/2018 07:09 AM, Florence Blanc-Renaud via FreeIPA-users wrote: >>>>>>> On 02/15/2018 11:47 AM, Bret Wortman via FreeIPA-users wrote: >>>>>>> >>>>>>>> On 02/15/2018 04:50 AM, Florence Blanc-Renaud wrote: >>>>>>>> On 02/15/2018 10:08 AM, Florence Blanc-Renaud via FreeIPA-users >>>>>>>> wrote: >>>>>>>>> On 02/14/2018 05:58 PM, Bret Wortman wrote: >>>>>>>>>>> On 02/14/2018 10:22 AM, Florence Blanc-Renaud wrote: >>>>>>>>>>>> On 02/14/2018 12:52 PM, Bret Wortman via FreeIPA-users wrote: >>>>>>>>>>>> I did figure out that I can use >>>>>>>>>>>> >>>>>>>>>>>> # ldapsearch -D 'directory manager' -W -E pr=20000 -b >>>>>>>>>>>> idnsname=damascusgrp.com,cn=dns,dc=damascusgrp,dc=com >>>>>>>>>>>> >>>>>>>>>>>> to list out all the entries, but the format isn't what I'm >>>>>>>>>>>> expecting. >>>>>>>>>>>> >>>>>>>>>>>> What I'm actually trying to do is move our whole >>>>>>>>>>>> infrastructure from one set of old & busted servers to some >>>>>>>>>>>> shiny new VMs. We'd like to extract the data and start fresh, >>>>>>>>>>>> as our replication agreements just don't seem to be working as >>>>>>>>>>>> expected. Changes to one don't always make it to the other and >>>>>>>>>>>> vice versa. While I'd love to dig in and solve that, it's >>>>>>>>>>>> easier right now to try to extract the data and reload it into >>>>>>>>>>>> a new server, build new replicas, then unbind & re-bind every >>>>>>>>>>>> client to the new server using ansible since we also lost our >>>>>>>>>>>> internal CA in the process. >>>>>>>>>>>> >>>>>>>>>>>> So while our current configuration is a mess, we can't afford >>>>>>>>>>>> to lose all the host/user/dns/hbac data in our servers. Thus, >>>>>>>>>>>> I've been capturing the output to text using various ipa >>>>>>>>>>>> *-find commands and have parsers to turn those back into new >>>>>>>>>>>> entries on the fresh hosts. DNS is the only thing that's >>>>>>>>>>>> holding me up. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Bret >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 02/14/2018 06:33 AM, Bret Wortman wrote: >>>>>>>>>>>>> Also, this doesn't solve the fact that the Web UI always >>>>>>>>>>>>> produces an error dialog whenever accessing our primary zone. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 02/13/2018 02:19 PM, Natxo Asenjo via FreeIPA-users wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Feb 13, 2018 at 8:13 PM, Natxo Asenjo >>>>>>>>>>>>>> <natxo.ase...@gmail.com <mailto:natxo.ase...@gmail.com>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> the canonical way to do this is using ldap paging, with >>>>>>>>>>>>>> ldapsearch you could try using the -E pr=xxxx >>>>>>>>>>>>>> parameter, where >>>>>>>>>>>>>> xxxx could be 1000 for instance. That way you know you >>>>>>>>>>>>>> are always >>>>>>>>>>>>>> under the limit imposed by the server. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> if you use -E pr=1000/noprompt, it will not prompt to >>>>>>>>>>>>>> continue, nicer for scripts obviously. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Groeten, >>>>>>>>>>>>>> natxo >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> FreeIPA-users mailing list >>>>>>>>>>>>>> --freeipa-users@lists.fedorahosted.org >>>>>>>>>>>>>> To unsubscribe send an email >>>>>>>>>>>>>> tofreeipa-users-le...@lists.fedorahosted.org >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> FreeIPA-users mailing list -- >>>>>>>>>>>> freeipa-users@lists.fedorahosted.org >>>>>>>>>>>> To unsubscribe send an email to >>>>>>>>>>>> freeipa-users-le...@lists.fedorahosted.org >>>>>>>>>>>> >>>>>>>>>>> Hi Bret, >>>>>>>>>>> >>>>>>>>>>> the search limits can be set at multiple levels: >>>>>>>>>>> - for the whole 389-ds server >>>>>>>>>>> nsslapd-sizelimit (in cn=config) >>>>>>>>>>> nsslapd-lookthroughlimit (in cn=config,cn=ldbm >>>>>>>>>>> database,cn=plugins,cn=config) >>>>>>>>>>> >>>>>>>>>>> - for operations performed through ipa * commands (or the webGUI): >>>>>>>>>>> ipaSearchRecordsLimit (in cn=ipaConfig,cn=etc,$BASEDN) >>>>>>>>>>> >>>>>>>>>>> - for each user: >>>>>>>>>>> nssizelimit and nsLookThroughLimit attributes (in >>>>>>>>>>> uid=$USER,cn=users,cn=accounts,$BASEDN) >>>>>>>>>>> >>>>>>>>>>> You are probably hitting one of these limits in your ipa *-find >>>>>>>>>>> command. >>>>>>>>>>> >>>>>>>>>>> HTH, >>>>>>>>>>> Flo >>>>>>>>>>> >>>>>>>>>> So I found almost all of these: >>>>>>>>>> >>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W -b 'cn=config' >>>>>>>>>> cn=config | grep nsslapd-sizelimit >>>>>>>>>> nsslapd-sizelimit: 2000 >>>>>>>>>> >>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W -b 'cn=config,cn=ldbm >>>>>>>>>> database,cn=plugins,cn=config' | grep lookthroughlimit >>>>>>>>>> nsslapd-lookthroughlimit: 100000 >>>>>>>>>> >>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W -b >>>>>>>>>> 'cn=ipaConfig,cn=etc,dc=damascusgrp,dc=com' | grep >>>>>>>>>> ipaSearchRecordsLimit >>>>>>>>>> ipaSearchRecordsLimit: 99999 >>>>>>>>>> >>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W -b >>>>>>>>>> 'uid=admin,cn=users,cn=accounts,dc=damascusgrp,dc=com' | grep -i >>>>>>>>>> limit >>>>>>>>>> (returns data but nothing matches) >>>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> nsSizeLimit and nsLookThroughLimit are operational attributes, >>>>>>>>> meaning that a standard ldapsearch will not return them. You need >>>>>>>>> either to specifically request them by providing them in the >>>>>>>>> attributes list: >>>>>>>>> >>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b $BASE nssizelimit >>>>>>>>> nslookthroughlimit >>>>>>>>> >>>>>>>>> or you can also specify + instead of the attributes in order to >>>>>>>>> get all operational attributes: >>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b $BASE + >>>>>>>>> >>>>>>>>> HTH, >>>>>>>>> Flo >>>>>>>>> >>>>>>>>>> The first doesn't seem to be something I can change. It's stuck >>>>>>>>>> at 2000, but since my issue occurs at 5000, I'm not worried >>>>>>>>>> about it. I believe that I'm missing something in the fourth >>>>>>>>>> search that might point me toward the attributes you mentioned >>>>>>>>>> but I'm not sure where. >>>>>>>>>> >>>>>>>> The 5000 limit rings a bell to me. It is the anonymous size limit. >>>>>>>> Can you check: >>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b cn=config -s base >>>>>>>> nsslapd-anonlimitsdn >>>>>>>> >>>>>>>> it will provide you with a DN of the entry defining the anonymous >>>>>>>> limits (usually cn=anonymous-limits,cn=etc,$BASEDN), then: >>>>>>>> >>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b >>>>>>>> cn=anonymous-limits,cn=etc,$BASEDN nsSizeLimit nsLookThroughLimit >>>>>>>> >>>>>>>> Now we should check the access log >>>>>>>> (/var/log/dirsrv/slapd-xxx/access) corresponding to your command >>>>>>>> to retrieve the DNS entries, and ensure which user identity is >>>>>>>> actually performing the search. >>>>>>>> >>>>>>>> Flo >>>>>>>>> _______________________________________________ >>>>>>>>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>>>>>>>> To unsubscribe send an email to >>>>>>>>> freeipa-users-le...@lists.fedorahosted.org >>>>>>> Both those limits are 5000. >>>>>>> >>>>>>> What should I be looking for in the access log? It gets a lot of >>>>>>> traffic and narrowing down exactly which entry represents the >>>>>>> original search is challenging. I'd rather not post the whole thing >>>>>>> because it's pretty large & noisy. >>>>>>> >>>>>> Hi, >>>>>> >>>>>> you can perform the ipa dnsrecord-find --all command, while tail'ing >>>>>> the access log. I will be interested in the lines containing the >>>>>> SEARCH with a err=4 (size limit exceeded). For this SEARCH, please >>>>>> note the connection number (conn=xx) and find the BIND for the same >>>>>> connection number (it will be located before the SEARCH). For each >>>>>> line (BIND and SEARCH) both the operation and the RESULT are >>>>>> interesting, for instance when doing ipa user-find: >>>>>> >>>>>> [date] conn=514 op=0 BIND dn="" method=sasl version=3 mech=GSS-SPNEGO >>>>>> [date] conn=514 op=0 RESULT err=0 tag=97 nentries=0 >>>>>> etime=0.0004754873 dn="uid=admin,cn=users,cn=accounts,$BASEDN" >>>>>> ... >>>>>> [date] conn=514 op=2 SRCH base="cn=users,cn=accounts,$BASEDN" >>>>>> scope=1 filter="(objectClass=posixaccount)" attrs="krbCanonicalName >>>>>> loginShell sshpubkeyfp gidNumber homeDirectory uidNumber uid >>>>>> givenName title sn krbPrincipalName mail nsAccountLock >>>>>> telephoneNumber ipaSshPubKey" >>>>>> [date] conn=514 op=2 RESULT err=0 tag=101 nentries=6 etime=0.0006313857 >>>>>> >>>>>> Flo >>>>>> >>>>>>> Bret >>>>>>> _______________________________________________ >>>>>>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>>>>>> To unsubscribe send an email to >>>>>>> freeipa-users-le...@lists.fedorahosted.org >>>>>> _______________________________________________ >>>>>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>>>>> To unsubscribe send an email to freeipa-users-leave@lists.fedorahos >>>>> Is this what you're after? >>>>> >>>>> conn=121474 op=0 SRCH base="" scope=0 filter="(objectClass=*)" >>>>> attrs="* altServer naming Contexts supportedControl >>>>> supportedExtension supportedFeatures supportedLDAPVersion >>>>> supportedSASLMechanisms domaincontrollerfunctionality >>>>> defaultnamingcontext lastusn highestcommittedusn aci" >>>>> conn=121474 op=0 RESULT err=0 tag=101 nentries=1 etime=0 >>>>> conn=121474 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI >>>>> conn=121474 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind >>>>> in progress >>>>> conn=121474 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI >>>>> conn=121474 op=2 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind >>>>> in progress >>>>> conn=121474 op=3 BIND dn="" method=sasl version=3 mech=GSSAPI >>>>> conn=121474 op=3 RESULT err=0 tag=101 nentries=0 etime=0 >>>>> dn="fqdn=loader4.my.net,cn=computers,cn=accoutns,dc=my,dc=net" >>>>> : >>>>> conn=121474 op=3033 SRCH base="cn=accounts,dc=my,dc=net" scope=2 >>>>> filter="(&(objectClass=ipaHost)(fqdn=sfile6.my.net))" >>>>> attrs="objectClass cn fqdn serverHostName memberOf ipaSshPubKey >>>>> ipaUniqueID" >>>>> conn=121474 op=3033 RESULT err=0 tag=101 nentries=1 etime=0 notes=P >>>>> pr_idx=0 pr_cookie=-1 >>>>> _______________________________________________ >>>>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>>>> To unsubscribe send an email to >>>>> freeipa-users-le...@lists.fedorahosted.org >>>> Hi, >>>> >>>> the command ipa dnsrecord-find should create a trace similar to the >>>> following: >>>> >>>> [date] conn=538 op=0 BIND dn="" method=sasl version=3 mech=GSS-SPNEGO >>>> [date] conn=538 op=0 RESULT err=0 tag=97 nentries=0 etime=0.0004778829 >>>> dn="uid=admin,cn=users,cn=accounts,$BASEDN" >>>> ... >>>> [date] conn=538 op=7 SRCH base="idnsname=$ZONE,cn=dns,$BASEDN" scope=2 >>>> filter="(&(objectClass=top)(objectClass=idnsrecord))" attrs="pTRRecord >>>> APLRecord SigRecord nAPTRRecord HIPRecord kXRecord aRecord aFSDBRecord >>>> hInfoRecord sRVRecord mDRecord certRecord nSECRecord idnsName >>>> tXTRecord DHCIDRecord dSRecord dNameRecord TLSARecord mXRecord >>>> sSHFPRecord cNAMERecord IPSECKEYRecord aAAARecord rRSIGRecord RPRecord >>>> DLVRecord URIRecord mInfoRecord KeyRecord a6Record nXTRecord LocRecord >>>> nSRecord SPFRecord" >>>> [date] conn=538 op=7 RESULT err=0 tag=101 nentries=11 etime=0.0005895601 >>>> >>>> In your case, since the search results are truncated, there will >>>> probably be err=4 (size limit exceeded) or err=11 (admin limit >>>> exceeded), and nentries= the limit you are hitting. >>>> >>>> Please keep in mind that 389-ds access log is buffered, it may get >>>> some time between the event and its logging in the file. >>>> >>>> Flo >>> Okay, found it. >>> >>> conn=128104 op=0 BIND dn="" method=sasl version=3 mech=GSS-SPNEGO >>> conn=128104 op=0 RESULT err=0 tag=97 nentries=0 etime=0 >>> dn="uid-admin,cn=users,cn=accoutns,dc=my,dc=net" >>> ... >>> conn=128104 op=7 SRCH base="idnsname=my.net,cn=dns,dc=my,dc=net" scope=2 >>> filter="(&(objectClass=top)(objectClass=idnsrecord))" attrs="sSHFPRecord >>> HIPRecord SPFRecord kXRecord nXTRecord mXRecord aAAARecord mDRecord >>> aRecord DLVReocrd TLSARecord * pTRRecord SigRecord idnsname aFSDBRecord >>> APLRecord URIRecord nAPTRRecord nSRecord LocRecord dNameRecord RPRecrod >>> DHCIDRecord IPSECKEYRecord rSIGRecord hInfoRecord cNAMERecord certRecord >>> sRVRecord dSRecord tXTRecord nSECRecord a6Record KeyRecord mInfoRecord >>> aci"" >>> conn=128104 op=7 RESULT err=11 tag=101 nentries=5000 etime=2 >>> >>> It was going to our other IPA server in the pair and I didn't notice >>> that I was doing my "ipa dnsrecord-find" from my workstation, not one of >>> the servers. >>> >>> So it looks like I'm getting an admin limit exceeded. What's the best >>> way to proceed? >> Hi, >> >> just to be sure, can you perform equivalent ldapsearch with GSS-SPNEGO / >> GSSAPI and plain bind: >> kinit admin >> ldapsearch -Y GSS-SPNEGO -b idnsname=my.net,cn=dns,dc=my,dc=net >> "(&(objectClass=top)(objectClass=idnsrecord))" >> >> kinit admin >> ldapsearch -Y GSSAPI -b idnsname=my.net,cn=dns,dc=my,dc=net >> "(&(objectClass=top)(objectClass=idnsrecord))" >> >> ldapsearch -D uid=admin,cn=users,cn=accounts,dc=my,dc=net -W -b >> idnsname=my.net,cn=dns,dc=my,dc=net >> "(&(objectClass=top)(objectClass=idnsrecord))" >> >> I would like to see if the auth mechanism has any impact on the limits >> and number of returned entries. >> Flo >> >>> _______________________________________________ >>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org >> _______________________________________________ >> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org >> _______________________________________________ >> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org > _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
