[
https://issues.apache.org/jira/browse/DIRSERVER-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14336308#comment-14336308
]
Emmanuel Lecharny commented on DIRSERVER-2047:
----------------------------------------------
So, let me give you some quick explaination about what those B-trees are good
for.
Storing entries into a LDAP server is not only about storing the entries
themselves in a B-tree. We of course do that, they are stored in what we call
the MasterTable. This table associates a unique UUID to a serialized entry. The
B-tree is ordered using the UUID.
Ok, now when doing a search you don't want to fetch all the entries from the
Mastertable, so we have external indexes, which are stored into other B-trees.
Typically, if you have indexed the 'cn' attribute, you will have a B-tree
associating each 'cn' with the list of entry UUID having this attribute.
There are a few mandatory indexes :
- objectClass
- entryCSN
- alias
- oneAlias
- present (used to index the attributeTypes that are used in ever entries)
- RDN
and a few others
An intersting index is the RDN index. In fact, this index uses 2 B-trees. Each
entry has a DN, which is a list of RDNs. We want to be able to retrieve an
entry using it's DN, and we use the RDN index for that purpose.
OTOH, the RDN index just have RDNs, not full DN. Add to that a RDN can be
shared between many entries. This index is a bit specific in that we store the
RDN plus it's parent's ID as a key. We have a <ParentIdAndRDN, UUID> B-tree
index for that.
The second B-TRree for the RDN index is a revert B-tree, ie a B-tree where the
UUID is the key and the ParentIdAndRDN the value. That allos us to build back
the DN which is not stored within the entry.
One could argue that the B-tree<UUID, ParentIdAndRdn> index is useless, because
we already have it : it's the MasterTable. That's true, except that we would
have to deserialize many Entries in order to rebuild the DN, which would be a
costly operation.
Hope it's helps...
> Some data can be lost when using ldapadd command to insert data into apacheds
> -----------------------------------------------------------------------------
>
> Key: DIRSERVER-2047
> URL: https://issues.apache.org/jira/browse/DIRSERVER-2047
> Project: Directory ApacheDS
> Issue Type: Bug
> Components: ldap
> Affects Versions: 2.0.0-M19
> Environment: centOS 6.5 with openldap and apacheds installed
> Reporter: linzhao
> Priority: Blocker
> Labels: LDAP
> Fix For: 2.0.0-M20
>
> Attachments: data.ldif, ordered-data.ldif, site.ldif, site_new.ldif,
> site_simple.ldif, site_topology_schema.ldif,
> site_topology_schema_simple.ldif, supercluster_partition.ldif, test.jpg
>
>
> In our system, we need to do data backup and restore for apacheds. For now,
> we use the ldapsearch and ldapadd command to do BR function. We use
> ldapsearch to backup apacheds data to be a ldif file and use ldapadd to
> restore the data. But when the ldif is a little big, I always found that the
> data can't be restored successfully, but the ladpadd command showed that the
> data can be added successfully. No exceptions for ldapadd command. But the
> restored data didn't exist in the node. This bug only happened when the ldif
> file is a little big. I mean if the data entry greater than 500 entries. But
> the node for backup and restore is using mavibot partition. Because I found
> so many problems for jdbm. So I change it to mavibot partition. Another
> question is that do you know is there some good way to do the data backup and
> restore for apacheds?
> May be my mavibot partition has some problems, so someone can tell me how to
> create a mavibot partition in APACHEDS? Because the mavibot parttion I
> created can't be viewed when click the "Open Configuration" in apacheds
> studio. Also I used unboundid JDK to insert many entries apacheds and the
> same problem happened. So someone can tell me how to config the mavibot
> partition on APACHEDS? That's may be very helpful.
> Below is the exceptions from apacheds when problems happened:
> [16:21:41] INFO [org.apache.directory.server.ldap.LdapServer] - Ldap service
> stopped.
> [16:21:41] WARN
> [org.apache.directory.server.core.shared.partition.DefaultPartitionNexus] -
> Failed to flush partition data out.
> org.apache.directory.api.ldap.model.exception.LdapOperationErrorException
> at
> org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.saveContextCsn(AbstractBTreePartition.java:3364)
> at
> org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.sync(DefaultPartitionNexus.java:319)
> at
> org.apache.directory.server.core.DefaultDirectoryService.shutdown(DefaultDirectoryService.java:1283)
> at org.apache.directory.server.ApacheDsService.stop(ApacheDsService.java:600)
> at
> org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.stop(ApacheDsTanukiWrapper.java:97)
> at org.tanukisoftware.wrapper.WrapperManager$13.run(WrapperManager.java:3134)
> Caused by: java.lang.NullPointerException
> at
> org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.saveContextCsn(AbstractBTreePartition.java:3350)
> ... 5 more
> [16:21:41] ERROR [org.apache.directory.server.wrapper.ApacheDsTanukiWrapper]
> - Failed to stop the service.
> org.apache.directory.api.util.exception.MultiException: ERR_265 Grouping many
> exceptions on root nexus sync()
> at
> org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.sync(DefaultPartitionNexus.java:328)
> at
> org.apache.directory.server.core.DefaultDirectoryService.shutdown(DefaultDirectoryService.java:1283)
> at org.apache.directory.server.ApacheDsService.stop(ApacheDsService.java:600)
> at
> org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.stop(ApacheDsTanukiWrapper.java:97)
> at org.tanukisoftware.wrapper.WrapperManager$13.run(WrapperManager.java:3134)
> Nested exceptions to follow:
> INFO | jvm 1 | 2015/02/02 17:11:06 | [17:11:06] WARN
> [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception
> forcing session to close: sending disconnect notice to client.
> INFO | jvm 1 | 2015/02/02 17:11:06 | java.io.IOException: Connection reset by
> peer
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> sun.nio.ch.IOUtil.read(IOUtil.java:197)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:311)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:45)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:694)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1121)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> INFO | jvm 1 | 2015/02/02 17:11:06 | at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
