thanks for the great work and more so for the detailed description, will try the code after you commit. On Wed, Jul 20, 2011 at 10:32 PM, Emmanuel Lecharny <[email protected]> wrote: > Hi guys, > > sorry for having been silent for a few days, I was playing locally with > Partitions. Let me give you some feedback on what I'm currently doing... > > The idea is to refactor the way we work with Partitions. It doe snot make > too much sense to have Partitions on one side and Store on another one, so I > have merged those two guys (it was separated last year, but Alex told me > that was a mistake). > > The current Partition and Store hierarchies are ([] = Abstract classes, () = > Interfaces, <> = concrete classes ) : > > (Partition) > [AbstractPartition] > [AbstractBTreePartition] > [AbstractLdifPartition] > <LdifPartition> > <ReadOnlyConfigurationPartition> > <SingleFileLdifPartition> > [AbstractXdbmPartition] > <AvlPartition> > <JdbmPartition> > <DefaultNexusPartition> > <SchemaPartition> > > (PartitionNexus) > <DefaultNexusPartition> > > (Store) > [AbstractStore] > <AvlStore> > <JdbmStore> > > Once we realize that the (Avl/jdbm)Partition are backed by an > (Avl/jdbm)Store, there is no reason not to have the classes merged. For > convenience, we can keep the Store interface (as it exposes some methods > specific to the backend. > > Now, we can go a bit further : the Ldif partitions are all encapsulating an > AvlPartition, because we load them in memory and it allows the system to > create a cursor on top of them. There is no real reason not to have the Ldif > partitions class inherit directly from the AvlPartition. That would give the > following hierarchy : > > (Partition) > [AbstractPartition] > [AbstractBTreePartition] > [AbstractXdbmPartition] > <AvlPartition> > [AbstractLdifPartition] > <LdifPartition> > <ReadOnlyConfigurationPartition> > <SingleFileLdifPartition> > <JdbmPartition> > <DefaultNexusPartition> > <SchemaPartition> > > (PartitionNexus) > <DefaultNexusPartition> > > As we keep the Store interface, we will make the > AbstractBTreePartitionimplement it : > > (Store) > [AbstractBTreePartition] > > The SchemaPartition and the DefaultNexusPartition are two different beasts : > - the SchemaPartition have a wrapped partition, which can be either a Ldif > partition, or a Jdbm partition, or whatever partition we want. > - the DefaultNexusPartition is just a proxy which reroute all the requests > to the associated partition, depending on the operation DN. > > Right now, this is what I'm having, and after a lot of cleanup, I have > something which is quite consistent. It allows us to easily add a new > Partition implementation §for instance, the HBase partition) > > Now, there are some slight issues that needed to be addressed : > - first, all those Partitions now take a SchemaManager as a constructor > parameter. This is not a problem, as the SchemaManager instance is always > created *before* we create any partition. > - second, partitions initialization was a nightmare. We had a > Store.init(SchemaManager) method, a final AbstractPartition.initialize() > method and an abstract AbstractPartition.doInit() method. I removed the > Store.init(SchemaManager), and refactored the doInti() methods in ordr to > have a proper initialization sequence (it's not perfect though). > > I do think we should merge the initialization() and doInit() method at some > point, I don't exactly know what is the best options atm. > > What remains to be done is to cleanup the DirectoryService initialization > and the DirectoryServiceFactory initialization. Basically, partitions > initialization is depending on a complex process, requiring many steps to be > done : > - setting the Id > - setting the SuffixDn > - creating the underlying storage if needed > - associating an optimizer, a cursorBuilder, an evaluatorBuilder and a > SearchEngine to the partition (for BTreePartitions) > - defining and initializing the indexes,creating the master table (for the > BTreePartitions) > - creating the contextEntry (for the LdifPartitions) > > Most of those steps are done in the doInit() methods, but it requires the > presence of configuration elements (Id, Indexes, SuffixDn, etc) which *have* > to be injected *before* we initialize the partition. > > The way we initialize the DirectoryService requires a lot of cleanup too. > For instance, the DefaultNexusPartition initialize the System partition, and > all the other partitions, which does not make a lot of sense to me. > Partitions should be associated with the DirectoryService, not with the > DefaultNexusPartition. For the same reason, the optimizer and searchEngine > should not be associated with a specific partition, as it makes thing way > more complex when we try to do search across partitions. > > One more thing would be to use a unique identifier across all the > partitions. Currently, we use a Long, starting at 1L for each partition's > contextEntry. Alex suggested that we use the entryUUID instead, something we > can do easily, as the classes are all generic (it uses ID). > > That's pretty much it. I'm working on my computer, because I was offline > last week-end, but I will create a branch and commit all my changes into it > so that you can land an eye. > > Feel free to provide any feedback ! > > > > > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com > >
-- Kiran Ayyagari
