OK thanks!! I included the text of the attachments in my response to you so I don't need to attach. Trying to get somewhere with this tonight. Is there a quick fail safe way to create an index? I noticed in my partition directory there are no .db file for the attributes I've indexed. I think your hunch is correct. If it has to wait until tomorrow This is ok. Thanks for all your help!!
Regards, Carlo Accorsi -----Original Message----- From: Emmanuel Lecharny [mailto:[email protected]] Sent: Wednesday, April 11, 2012 2:43 PM To: [email protected] Subject: RE: Plea for help with search performance ok. i'll check that tomorrow. the brst you can do is to fill a JIRA wjere you cam attach files. be aware that those infos will be public. Le 11 avr. 2012 19:34, <[email protected]> a écrit : > Emmanuel ok great.. I just responded to Kiran and realized the > attachments don't come through. > > I create everything with ldif files, restarting in between them. > I start with a clean install, star the default instance once, the > import the first file. > > ## begin file 1## > #version: 1 > > dn: > ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=c > onfig > objectclass: top > objectClass: ads-base > objectclass: ads-partition > objectclass: ads-jdbmPartition > ads-indexes: apacheRdn > ads-indexes: apacheSubLevel > ads-indexes: apachePresence > ads-indexes: apacheOneLevel > ads-indexes: apacheOneAlias > ads-indexes: apacheSubAlias > ads-indexes: apacheAlias > ads-indexes: entryUUID > ads-indexes: entryCSN > ads-indexes: o > ads-indexes: krb5PrincipalName > ads-indexes: objectClass > ads-indexes: ou > ads-indexes: uid > ads-indexes: employeeNumber > ads-indexes: displayName > ads-indexes: cn > ads-indexes: mail > ads-partitionSuffix: o=cpro > ads-jdbmpartitionoptimizerenabled: TRUE > ads-partitioncachesize: 50000 > ads-partitionsynconwrite: TRUE > ads-partitionid: cpro > ads-enabled: TRUE > > dn: > ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=d > efault,ou=config > objectclass: organizationalUnit > objectclass: top > ou: indexes > > dn: > ads-indexAttributeId=apacheRdn,ou=indexes,ads-partitionId=cpro,ou=part > itions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apacheRdn > ads-enabled: TRUE > ads-indexcachesize: 5000 > > dn: > ads-indexAttributeId=apacheSubLevel,ou=indexes,ads-partitionId=cpro,ou > =partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apacheSubLevel > ads-enabled: TRUE > ads-indexcachesize: 5000 > > dn: > ads-indexAttributeId=apachePresence,ou=indexes,ads-partitionId=cpro,ou > =partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apachePresence > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=apacheOneLevel,ou=indexes,ads-partitionId=cpro,ou > =partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apacheOneLevel > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=apacheOneAlias,ou=indexes,ads-partitionId=cpro,ou > =partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apacheOneAlias > ads-enabled: TRUE > ads-indexcachesize: 5000 > > dn: > ads-indexAttributeId=apacheSubAlias,ou=indexes,ads-partitionId=cpro,ou > =partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apacheSubAlias > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=apacheAlias,ou=indexes,ads-partitionId=cpro,ou=pa > rtitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: apacheAlias > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=o,ou=indexes,ads-partitionId=cpro,ou=partitions,a > ds-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: o > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=krb5PrincipalName,ou=indexes,ads-partitionId=cpro > ,ou=partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: krb5PrincipalName > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=objectClass,ou=indexes,ads-partitionId=cpro,ou=pa > rtitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: objectClass > ads-enabled: TRUE > ads-indexcachesize: 50000 > > dn: > ads-indexAttributeId=entryUUID,ou=indexes,ads-partitionId=cpro,ou=part > itions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: entryUUID > ads-enabled: TRUE > ads-indexcachesize: 5000 > > dn: > ads-indexAttributeId=entryCSN,ou=indexes,ads-partitionId=cpro,ou=parti > tions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: entryCSN > ads-enabled: TRUE > ads-indexcachesize: 5000 > > dn: > ads-indexAttributeId=ou,ou=indexes,ads-partitionId=cpro,ou=partitions, > ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: ou > ads-enabled: TRUE > ads-indexcachesize: 100 > > dn: > ads-indexAttributeId=uid,ou=indexes,ads-partitionId=cpro,ou=partitions > ,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: uid > ads-enabled: TRUE > ads-indexcachesize: 50000 > > dn: > ads-indexAttributeId=employeeNumber,ou=indexes,ads-partitionId=cpro,ou > =partitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: employeeNumber > ads-enabled: TRUE > ads-indexcachesize: 50000 > > dn: > ads-indexAttributeId=displayName,ou=indexes,ads-partitionId=cpro,ou=pa > rtitions,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-indexattributeid: displayName > ads-enabled: TRUE > ads-indexcachesize: 50000 > > dn: > ads-indexAttributeId=cn,ou=indexes,ads-partitionId=cpro,ou=partitions, > ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-base > objectclass: ads-jdbmIndex > objectclass: top > ads-indexattributeid: cn > ads-enabled: TRUE > ads-indexcachesize: 50000 > > dn: > ads-indexAttributeId=mail,ou=indexes,ads-partitionId=cpro,ou=partition > s,ads-directoryServiceId=default,ou=config > objectclass: ads-index > objectclass: ads-base > objectclass: ads-jdbmIndex > objectclass: top > ads-indexattributeid: mail > ads-enabled: TRUE > ads-indexcachesize: 50000 > > #Change password policy to allow hashed passwords from ldif > dn: > ads-pwdId=default,ou=passwordPolicies,ads-interceptorId=authentication > Interceptor,ou=interceptors,ads-directoryServiceId=default,ou=config > changetype: modify > replace: ads-pwdcheckquality > ads-pwdcheckquality: 0 > > ## end file 1 ### > > Then I restart the server and re connect > > ## begin file 2 ## > #version: 1 > > dn: o=cpro > objectClass: organization > objectClass: top > o: cpro > > dn: ou=int,o=cpro > objectClass: organizationalUnit > objectClass: top > ou: int > description: Internal > > dn: ou=users,ou=int,o=cpro > objectClass: organizationalUnit > objectClass: top > ou: users > description: Internal Users > > dn: ou=groups,ou=int,o=cpro > objectClass: organizationalUnit > objectClass: top > ou: groups > description: Internal Groups > > dn: ou=ext,o=cpro > objectClass: organizationalUnit > objectClass: top > ou: ext > description: External > > dn: ou=users,ou=ext,o=cpro > objectClass: organizationalUnit > objectClass: top > ou: users > description: External Users > > #External Password policy > dn: > ads-pwdId=cproext,ou=passwordPolicies,ads-interceptorId=authentication > Interceptor,ou=interceptors,ads-directoryServiceId=default,ou=config > objectclass: top > objectclass: ads-base > objectclass: ads-passwordPolicy > ads-pwdattribute: userPassword > ads-pwdid: cproext > ads-enabled: TRUE > ads-pwdallowuserchange: TRUE > ads-pwdcheckquality: 0 > ads-pwdexpirewarning: 600 > ads-pwdfailurecountinterval: 30 > ads-pwdgraceauthnlimit: 5 > ads-pwdgraceexpire: 0 > ads-pwdinhistory: 5 > ads-pwdlockout: TRUE > ads-pwdlockoutduration: 0 > ads-pwdmaxage: 0 > ads-pwdmaxdelay: 0 > ads-pwdmaxfailure: 5 > ads-pwdmaxidle: 0 > ads-pwdmaxlength: 0 > ads-pwdminage: 0 > ads-pwdmindelay: 0 > ads-pwdminlength: 5 > ads-pwdmustchange: FALSE > ads-pwdsafemodify: FALSE > > dn: > ads-pwdId=cproint,ou=passwordPolicies,ads-interceptorId=authentication > Interceptor,ou=interceptors,ads-directoryServiceId=default,ou=config > objectclass: ads-passwordPolicy > objectclass: ads-base > objectclass: top > ads-pwdattribute: userPassword > ads-pwdid: cproint > ads-enabled: TRUE > ads-pwdallowuserchange: TRUE > ads-pwdcheckquality: 0 > ads-pwdexpirewarning: 10 > ads-pwdfailurecountinterval: 30 > ads-pwdgraceauthnlimit: 5 > ads-pwdgraceexpire: 0 > ads-pwdinhistory: 5 > ads-pwdlockout: TRUE > ads-pwdlockoutduration: 0 > ads-pwdmaxage: 60 > ads-pwdmaxdelay: 0 > ads-pwdmaxfailure: 5 > ads-pwdmaxidle: 0 > ads-pwdmaxlength: 0 > ads-pwdminage: 0 > ads-pwdmindelay: 0 > ads-pwdminlength: 10 > ads-pwdmustchange: FALSE > ads-pwdsafemodify: FALSE > > ## end file 2 ## > > I then update the partition cache to 50000 and restart again , before > importing any users. > Thanks. > > Regards, > Carlo Accorsi > > > > -----Original Message----- > From: Emmanuel Lécharny [mailto:[email protected]] > Sent: Wednesday, April 11, 2012 1:17 PM > To: [email protected] > Subject: Re: Plea for help with search performance > > Ok, > > after some investigation, I'm pretty sure that the indexes are not > created. When I create the indexes, I get those results : > > - 80000 entries injected into the server, with uid=<number> > - displayName is indexed, uid is not indexed > > - first search : > connection.search("dc=example,dc=com", "(displayName=12345Awg-Rosli, > Awg-Abd-Rahim SMDS-UIA/G/MMO52D)", SearchScope.SUBTREE, "*" ); > > result : > Delta search : 16 ms > entry found : > dn[n]: uid=12345,dc=example,dc=com > objectclass: organizationalPerson > objectclass: person > objectclass: inetOrgPerson > objectclass: top > title: Snr Operations Technician (D) > uid: 12345 > description: UI - S > businesscategory: Ops MDS (Malaysia) Sdn Bhd > departmentnumber: SMDS - UIA/G/MMO52D > employeenumber: A-A-R.Awg-Rosli > givenname: Awg-Abd-Rahim > cn: Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D > sn: Awg-Rosli > telephonenumber: 555-1212 > mail: [email protected] > displayname: 12345Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D > > - Second search : > connection.search("dc=example,dc=com", "(displayName=34567*)", > SearchScope.SUBTREE, "*" ); > > result : > Delta search substring : 9 ms > entry found : > dn[n]: uid=34567,dc=example,dc=com > objectclass: organizationalPerson > objectclass: person > objectclass: inetOrgPerson > objectclass: top > title: Snr Operations Technician (D) > uid: 34567 > description: UI - S > businesscategory: Ops MDS (Malaysia) Sdn Bhd > departmentnumber: SMDS - UIA/G/MMO52D > employeenumber: A-A-R.Awg-Rosli > givenname: Awg-Abd-Rahim > cn: Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D > sn: Awg-Rosli > telephonenumber: 555-1212 > mail: [email protected] > displayname: 34567Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D > > -third search : > connection.search("dc=example,dc=com", "(uid=67890)", > SearchScope.SUBTREE, "*" ); > > result : > Delta search no index : 38985 ms > > entry found : > dn[n]: uid=67890,dc=example,dc=com > objectclass: organizationalPerson > objectclass: person > objectclass: inetOrgPerson > objectclass: top > title: Snr Operations Technician (D) > uid: 67890 > description: UI - S > businesscategory: Ops MDS (Malaysia) Sdn Bhd > departmentnumber: SMDS - UIA/G/MMO52D > employeenumber: A-A-R.Awg-Rosli > givenname: Awg-Abd-Rahim > cn: Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D > sn: Awg-Rosli > telephonenumber: 555-1212 > mail: [email protected] > displayname: 67890Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D > > > So if the index is properly set, the search takes milliseconds to > complete (which is expected). Otherwise, it can take tens of seconds... > > Two possibilities : > - you don't initialize the index correctly. Please provide your > configuration and if you are using ADS embedded, please provide the > code that you use to embed it > - we have an issue in the way we initialize the index, which is more > likely to be the problem, as I fixed one issue in this area 2 weeks > ago (in > M7-SNAPSHOT) and I just fixed another one while doing the experiment... > > I'd like to rule out the first hypothesis first, but in any case, I'll > do some more check tonite or tomorrow. > > > Many thanks for your patience. > > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com > >
