ads-partitionSyncOnWrite=FALSE did the trick! Back to 80 adds/sec, Thank you!!
Regards, Carlo Accorsi -----Original Message----- From: Emmanuel Lécharny [mailto:[email protected]] Sent: Wednesday, October 10, 2012 3:16 AM To: [email protected] Subject: Re: build monday from trunk very slow for ldif import Le 10/10/12 4:55 AM, [email protected] a écrit : > Hi, I'm still trying finish the testing of all the Password policy work Karin > did over the weekend but I have another issue that's come up. Ldif imports > are extremely slow. > > During our testing, we often delete the entire partition directory to start > fresh. When the server starts, it lays down the partition and .db files as > defined config.ldif. > > Anyway, we used to import (via studio) an ldif file with 80k entries and it > would load about 90 entries per second. That was great! > With this build it's going at about 4-5 entries per second. Hmmm. This is very slow. How many indexed attributes do you have ? I have done some tests locally, and I'm able to get up to 200 add/s, but with a simpler entry. > > We noticed is that previously, the partition .db files would not change (on > disk) until after the ldif import was complete. > Then when we stopped the server, it was like the entire import got flushed to > the disk at once. The files would go from 20K to 400MB. > With this build, it seems to be updating the files as it goes. Could this be > the reason? It may be because we flush on disk for every single write. You could turn the ads-partitionSyncOnWrite flag to FALSE, so that the data are flush only ever ads-dsSyncPeriodMillis (default to 15 seconds) > > Also, this is the first build I noticed the .lg files in the partition > directory. I think they're there for journaling but don't know if that's an > option something new? It's a JDBM file which is created beside the db files, AFAIR. I have to double check that. > > We removed all the password policy Attributes from my ldif file thinking that > was slowing it down but it's essentially the same performance. > Below is my partition and all the indexes are set like the one I included. > Any changes that would affect this in the last few weeks. Anyone else seeing > this? Thanks! > > dn: > ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=c > onfig > objectclass: top > objectClass: ads-base > objectclass: ads-partition > objectclass: ads-jdbmPartition > ads-indexes: apacheRdn > ads-indexes: apacheSubLevel > ads-indexes: apachePresence > ads-indexes: apacheOneLevel > ads-indexes: apacheOneAlias > ads-indexes: apacheSubAlias > ads-indexes: apacheAlias > ads-indexes: entryCSN > ads-indexes: krb5PrincipalName > ads-indexes: objectClass > ads-indexes: ou > ads-indexes: uid > ads-indexes: employeeNumber > ads-indexes: displayName > ads-indexes: cn > ads-indexes: mail > ads-indexes: roomNumber > ads-indexes: pwdPolicySubEntry > ads-indexes: member > ads-indexes: description > ads-indexes: givenName > ads-indexes: sn > ads-indexes: administrativeRole > ads-partitionSuffix: o=cpro > ads-jdbmpartitionoptimizerenabled: TRUE > ads-partitioncachesize: 100 > ads-partitionsynconwrite: TRUE > ads-partitionid: cpro > ads-enabled: TRUE > > #index example, they're all like this..HasReverse=FALSE > dn: > ads-indexAttributeId=uid,ou=indexes,ads-partitionId=cpro,ou=partitions > ,ads-directoryServiceId=default,ou=config > ads-indexattributeid: uid > ads-indexHasReverse: FALSE > ads-indexcachesize: 100 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
