[jira] [Commented] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035118#comment-17035118 ] Rashid Mahmood commented on DIRSERVER-2299: --- I already mentioned in issue description which version we are using. If new version is using one .db file then why repair command generates a zero byte size no named .db file, see 4th comment. I still dont understand why index information was lost after server stop, possible reason could be that indexes were never written on file-system and were kept in memory. Is there any tool from ApacheDS to validate the database for corruption and to explore the meta information inside database? We are afraid that in production environment we can face severe search issues (because of indexing) or possible data corruption issues with data replication. Is ApacheDS Production Ready? > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: ApacheDS 2.0.0.AM24-example-structure.png, ApacheDS > 2.0.0.AM25-example-structure.png, after-repair.png, > image-2020-02-06-16-53-48-839.png, image-2020-02-11-17-02-41-811.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Commented] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035053#comment-17035053 ] Emmanuel Lécharny commented on DIRSERVER-2299: -- The newest version will not create a file per index, because we use transaction across B-trees that forces the data to be flushed all in one when committing the transaction, using one single RecordManager - thus one file -. > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: ApacheDS 2.0.0.AM24-example-structure.png, ApacheDS > 2.0.0.AM25-example-structure.png, after-repair.png, > image-2020-02-06-16-53-48-839.png, image-2020-02-11-17-02-41-811.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
Re: Importing a backup of data
On 11/02/2020 14:40, Davies, Matt wrote: Thanks Emmanuel I've got to that bit, and I'm seeing loads of data now, much much better The only issue now is that it's not in the correct order, and it's failing creating items, when their parent item hasn't been created yet but is further down in the file. We're not passing a -S option, but we are passing a -s, but I don't think that relates to the order of the results, whilst -S does. Any ideas what switch to use to ensure the results follow the correct parent first pattern so I can import it? Sadly, there is no such option that enforces the ordering the way you want. The only solution is to inject the file until it get accepted fully, discarding the errors (ldapadd -c ...). Otherwise, you have to massage the file 'by hand'. Ldap is not necessarily funny... - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Comment Edited] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034568#comment-17034568 ] Rashid Mahmood edited comment on DIRSERVER-2299 at 2/11/20 4:19 PM: I think version AM25 introduced some bug in index handling, see this is the example partition structure in AM25 !ApacheDS 2.0.0.AM25-example-structure.png! this is the example partition structure in M24 which seems to be the same layout what you have written in first comment !ApacheDS 2.0.0.AM24-example-structure.png! was (Author: ramahmoo): I think version AM25 created some bug in index handling, see this is the example partition structure in AM25 !ApacheDS 2.0.0.AM25-example-structure.png! this is the example partition structure in AM24 which seems to be the same layout what you have written in first comment !ApacheDS 2.0.0.AM24-example-structure.png! > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: ApacheDS 2.0.0.AM24-example-structure.png, ApacheDS > 2.0.0.AM25-example-structure.png, after-repair.png, > image-2020-02-06-16-53-48-839.png, image-2020-02-11-17-02-41-811.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Commented] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034568#comment-17034568 ] Rashid Mahmood commented on DIRSERVER-2299: --- I think version AM25 created some bug in index handling, see this is the example partition structure in AM25 !ApacheDS 2.0.0.AM25-example-structure.png! this is the example partition structure in AM24 which seems to be the same layout what you have written in first comment !ApacheDS 2.0.0.AM24-example-structure.png! > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: ApacheDS 2.0.0.AM24-example-structure.png, ApacheDS > 2.0.0.AM25-example-structure.png, after-repair.png, > image-2020-02-06-16-53-48-839.png, image-2020-02-11-17-02-41-811.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Updated] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rashid Mahmood updated DIRSERVER-2299: -- Attachment: ApacheDS 2.0.0.AM24-example-structure.png > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: ApacheDS 2.0.0.AM24-example-structure.png, ApacheDS > 2.0.0.AM25-example-structure.png, after-repair.png, > image-2020-02-06-16-53-48-839.png, image-2020-02-11-17-02-41-811.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Updated] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rashid Mahmood updated DIRSERVER-2299: -- Attachment: ApacheDS 2.0.0.AM25-example-structure.png > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: ApacheDS 2.0.0.AM25-example-structure.png, > after-repair.png, image-2020-02-06-16-53-48-839.png, > image-2020-02-11-17-02-41-811.png, index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Updated] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rashid Mahmood updated DIRSERVER-2299: -- Attachment: image-2020-02-11-17-02-41-811.png > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: after-repair.png, image-2020-02-06-16-53-48-839.png, > image-2020-02-11-17-02-41-811.png, index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Commented] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034449#comment-17034449 ] Rashid Mahmood commented on DIRSERVER-2299: --- I have tested now with 9th Feb Snapshot from [https://repository.apache.org/content/groups/snapshots/org/apache/directory/server/apacheds-service/2.0.0.AM26-SNAPSHOT/] I see no difference in behaviour. > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: after-repair.png, image-2020-02-06-16-53-48-839.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
RE: Importing a backup of data
Thanks Emmanuel I've got to that bit, and I'm seeing loads of data now, much much better The only issue now is that it's not in the correct order, and it's failing creating items, when their parent item hasn't been created yet but is further down in the file. We're not passing a -S option, but we are passing a -s, but I don't think that relates to the order of the results, whilst -S does. Any ideas what switch to use to ensure the results follow the correct parent first pattern so I can import it? Thanks again Emmanuel From: Emmanuel Lécharny [elecha...@gmail.com] Sent: 11 February 2020 13:11 To: dev@directory.apache.org Subject: Re: Importing a backup of data On 11/02/2020 11:24, Davies, Matt wrote: > Hi there > > A colleague of mine recently exported what we believed to be all the data > from a running apacheds v24 server like so > > ldapsearch -h hostname -x -p 10389 -b "dc=org" -s sub "(ObjectClass=)" + > > backup_060220.ldif > > I've then ran through a number of ldif scripts, adding them to a new > installation to build up the same structure on a different machine, but when > I try to add the backup file it's failing right at the start > > ldapadd -h localhost -p 10389 -x -w [PASSWORD] -D "uid=admin,ou=system" -f > /tmp/backup_060220.ldif > > adding new entry "dc=DansOrg2,dc=org" > ldap_add: Object class violation (65) > additional info: OBJECT_CLASS_VIOLATION: failed for MessageType : > ADD_REQUEST > Message ID : 2 > Add Request : > Entry > dn[n]: dc=DansOrg2,dc=org > > subschemaSubentry: cn=schema > createTimestamp: 20191018141712.279Z > entryUUID: a98fa576-d440-49d5-b336-5f175a5c619e > entryDN: dc=DansOrg2,dc=org > nbSubordinates: 2 > entryCSN: 20191018141712.279000Z#00#001#00 > entryParentId: 2115906e-3e4d-433e-95f3-3311038027b2 > nbChildren: 2 > creatorsName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system > : ERR_60 Entry dc=DansOrg2,dc=org does not contain a STRUCTURAL ObjectClass > > > Looking at the backup ldif file, there are numerous entries at the beginning > that don't have any objectclass defined, here's the first one > > # DansOrg2.org > dn: dc=DansOrg2,dc=org > entryCSN: 20191018141712.279000Z#00#001#00 > nbSubordinates: 2 > nbChildren: 2 > createTimestamp: 20191018141712.279Z > creatorsName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system > subschemaSubentry: cn=schema > entryDN: dc=DansOrg2,dc=org > entryUUID: a98fa576-d440-49d5-b336-5f175a5c619e > entryParentId: 2115906e-3e4d-433e-95f3-3311038027b2 > > We're very new to apacheds, so I don’t know if we're creating the backup > incorrectly, or importing it incorrectly, or a bit of both. > > I don't really understand how it could import without an object class. You specified that you wanted only the operational attributes in your search request ( the '+' at the end of ...-s sub "(ObjectClass=)" +) Replace it with '*', that should do the trick. Or use '*' and '+'. Note : you should normally not specify either of them. The operational attributes are internal attributes that are not necessarily needed (except in some corner cases) and teh default is for a search to return all the other attributes (ie you don't have to add the '*') Hope it helps. Emmanuel - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Commented] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034432#comment-17034432 ] Emmanuel Lécharny commented on DIRSERVER-2299: -- I suspect this has been fixed in trunk, I'm currently working on cutting a release this week. That will take a few days. > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: after-repair.png, image-2020-02-06-16-53-48-839.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
Re: Importing a backup of data
On 11/02/2020 11:24, Davies, Matt wrote: Hi there A colleague of mine recently exported what we believed to be all the data from a running apacheds v24 server like so ldapsearch -h hostname -x -p 10389 -b "dc=org" -s sub "(ObjectClass=)" + > backup_060220.ldif I've then ran through a number of ldif scripts, adding them to a new installation to build up the same structure on a different machine, but when I try to add the backup file it's failing right at the start ldapadd -h localhost -p 10389 -x -w [PASSWORD] -D "uid=admin,ou=system" -f /tmp/backup_060220.ldif adding new entry "dc=DansOrg2,dc=org" ldap_add: Object class violation (65) additional info: OBJECT_CLASS_VIOLATION: failed for MessageType : ADD_REQUEST Message ID : 2 Add Request : Entry dn[n]: dc=DansOrg2,dc=org subschemaSubentry: cn=schema createTimestamp: 20191018141712.279Z entryUUID: a98fa576-d440-49d5-b336-5f175a5c619e entryDN: dc=DansOrg2,dc=org nbSubordinates: 2 entryCSN: 20191018141712.279000Z#00#001#00 entryParentId: 2115906e-3e4d-433e-95f3-3311038027b2 nbChildren: 2 creatorsName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system : ERR_60 Entry dc=DansOrg2,dc=org does not contain a STRUCTURAL ObjectClass Looking at the backup ldif file, there are numerous entries at the beginning that don't have any objectclass defined, here's the first one # DansOrg2.org dn: dc=DansOrg2,dc=org entryCSN: 20191018141712.279000Z#00#001#00 nbSubordinates: 2 nbChildren: 2 createTimestamp: 20191018141712.279Z creatorsName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system subschemaSubentry: cn=schema entryDN: dc=DansOrg2,dc=org entryUUID: a98fa576-d440-49d5-b336-5f175a5c619e entryParentId: 2115906e-3e4d-433e-95f3-3311038027b2 We're very new to apacheds, so I don’t know if we're creating the backup incorrectly, or importing it incorrectly, or a bit of both. I don't really understand how it could import without an object class. You specified that you wanted only the operational attributes in your search request ( the '+' at the end of ...-s sub "(ObjectClass=)" +) Replace it with '*', that should do the trick. Or use '*' and '+'. Note : you should normally not specify either of them. The operational attributes are internal attributes that are not necessarily needed (except in some corner cases) and teh default is for a search to return all the other attributes (ie you don't have to add the '*') Hope it helps. Emmanuel - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
[jira] [Commented] (DIRSERVER-2299) Missing Indexes after Restart
[ https://issues.apache.org/jira/browse/DIRSERVER-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034335#comment-17034335 ] Rashid Mahmood commented on DIRSERVER-2299: --- Any Update on following? * verify that for every index, an index-name.db file is created * repair works as it is supposed to work * why on stop indexes are lost > Missing Indexes after Restart > - > > Key: DIRSERVER-2299 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2299 > Project: Directory ApacheDS > Issue Type: Bug > Components: index, ldap >Affects Versions: 2.0.0.AM25 >Reporter: Rashid Mahmood >Priority: Major > Attachments: after-repair.png, image-2020-02-06-16-53-48-839.png, > index-config.png > > > We have decided to use ApacheDS (version 2.0.0.AM25) for Authentication in > our product. Ldap Server and Directory Service are being used in Embedded > Mode. > > During our Tests we found that after Restart, some of the Entries via Indexed > Attribute Search Filter are not comming in search result. After considering > various possibilities we found that it is Indexing problem. We used Repair > commnd and then we see those entries in search results. > > Why some of the indexes were missing even it was a gracefull shutdown? > > We want to avoid repair command after every restart unless it is really > necessary. Our product will serve in future more than 50K users, we have fear > that it would cost us a lot if we encounter those unexpected problems. > > These are two attributes used in our search filters > dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: mail > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > dn: ads-indexAttributeId=telID,ou=indexes,ads-partitionId=tsi,ou=part > itions,ads-directoryServiceId=default,ou=config > ads-indexattributeid: telID > ads-indexHasReverse: FALSE > ads-indexcachesize: 1000 > objectclass: ads-index > objectclass: ads-jdbmIndex > objectclass: ads-base > objectclass: top > ads-enabled: TRUE > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@directory.apache.org For additional commands, e-mail: dev-h...@directory.apache.org
Importing a backup of data
Hi there A colleague of mine recently exported what we believed to be all the data from a running apacheds v24 server like so ldapsearch -h hostname -x -p 10389 -b "dc=org" -s sub "(ObjectClass=)" + > backup_060220.ldif I've then ran through a number of ldif scripts, adding them to a new installation to build up the same structure on a different machine, but when I try to add the backup file it's failing right at the start ldapadd -h localhost -p 10389 -x -w [PASSWORD] -D "uid=admin,ou=system" -f /tmp/backup_060220.ldif adding new entry "dc=DansOrg2,dc=org" ldap_add: Object class violation (65) additional info: OBJECT_CLASS_VIOLATION: failed for MessageType : ADD_REQUEST Message ID : 2 Add Request : Entry dn[n]: dc=DansOrg2,dc=org subschemaSubentry: cn=schema createTimestamp: 20191018141712.279Z entryUUID: a98fa576-d440-49d5-b336-5f175a5c619e entryDN: dc=DansOrg2,dc=org nbSubordinates: 2 entryCSN: 20191018141712.279000Z#00#001#00 entryParentId: 2115906e-3e4d-433e-95f3-3311038027b2 nbChildren: 2 creatorsName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system : ERR_60 Entry dc=DansOrg2,dc=org does not contain a STRUCTURAL ObjectClass Looking at the backup ldif file, there are numerous entries at the beginning that don't have any objectclass defined, here's the first one # DansOrg2.org dn: dc=DansOrg2,dc=org entryCSN: 20191018141712.279000Z#00#001#00 nbSubordinates: 2 nbChildren: 2 createTimestamp: 20191018141712.279Z creatorsName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system subschemaSubentry: cn=schema entryDN: dc=DansOrg2,dc=org entryUUID: a98fa576-d440-49d5-b336-5f175a5c619e entryParentId: 2115906e-3e4d-433e-95f3-3311038027b2 We're very new to apacheds, so I don’t know if we're creating the backup incorrectly, or importing it incorrectly, or a bit of both. I don't really understand how it could import without an object class. Any help or advice would be very welcome. Thanks Matt
