On Fri, Jan 20, 2012 at 9:47 AM, Emmanuel Lecharny <[email protected]> wrote: > Hi guys, > > I was a bit silent last week and this week, let me update you about what I > was working on. > > First of all, I have had to deal with some familly issues, which ate half of > my time. > > Regarding the Txn branch I was working on until last week, I stopped because > I was not able to fix the code without a serious help from Selcuk. As he is > busy, I preferred to wait for him to be available again, instead of bullying > into the code and break it seriously. I believe that there are some > improvement since the moment I started to work on the branch, but it's not > working fully yet.
I am starting to look into this again. I did an svn up and txn manager tests in core.shared are not working. Do you have the same problem? Also what was the test you thought was returning incorrect results? I couldnt find the relevant email. > > So I switched to something we wanted to do a long time ago : designing a new > version of JDBM. JDBM is a BTree implementation, with locks to protect > concurrent access. The idea was to implement a MVCC solution on top of a > BTree : > - each search can be done concurrently with any other operation, because it > asks for a specific existing revision from the btree > - each modification is done on a new revision > - two modifications can't be done at the same time (so modifications are > queued and executed one after the other) > > The consequence is that searches will be very fast. It comes to a price > though : we keep a track on every revision, until it's not used anymore. > This is done by copying every modified pages when applying some > modification. > > As of today, the addition operation and the find operation is working just > fine. I conducted some benchmark on additions, and it seems that the system > is pretty decent. > > A *lot* remains to be done : > - deletion must be implemented > - browsing the tree is not yet implemented > - it's all in memory atm > - we must add some semaphore for concurrent modifications > - a GC must be added to discard unused pages > > But most of all, as it's a in-memory btree atm, I must add the disk layer. > It will be based on Memory Mapped files. > > Once those preliminary works will be done, the idea is to use this > implementation to replace JDBM. That would make the server consistent, and > we may then use it without the in-memory txn layer. > > Not to say that this txn layer is useless; using a MVCC btree based backend > is *not* enough : we have no way to guarantee the atomicity of move > operation across partitions. > > This work has been done in my sandbox, where you can follow the work in > progress : > http://svn.apache.org/viewvc/directory/sandbox/elecharny/shared-mvbt > > At the same time, thanks to Pierre-Arnaud, a first milstone of Studio 2.0 > has been released, and it exposed some nasty bugs in the LDAP API. Which is > actually a good thing : we can fix them ! > > So keep tuned, a lot of new things are coming soon ! > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com > thanks Selcuk
