Hi guys,
sorry for having been silent those last 10 days, I had many private
things to manage, plus I wantd to take a bit of a break to calmy think
about our options.
I had a long convo with Alex last week (or the week before, I don't
remember), about the problem we had with the merge, and the various
issues we had with the txn layer (at least, the pb I was perceiving when
I had this convo). Bottom line, I think I freaked out about the
coplexity of th emerge, plus the fact that some serious point needed to
be handled if we switch to the txn layer.
Here is what I think as of today :
- there is no mean to merge the branch into the trunk : it will be way
easier to do the opposite, simply because I *know* what I changed in the
trunk, also because I have already merged the trunk into the branch many
times, so it will be easier to do so
- it will take a bit of time, but that worth the effort
- in any case, the txn-layer is the oly exit strategy we have right now.
I have tried to think about a better solution that would help us in a
very short delay, but there is none.
- the problem we have in the txn layer can be mitigated with time, and
we have plenty, as soon as we can deliver a server that does not have
concurrent access issues
Now, we do have some concerns to take care of in the near future, if the
txn branch becomes the new trunk :
- the fact that we handle MVCC in memory is problematic, as we may keep
in memory a hell lots of page : a single modification will copy log(N)
pages in memory, if N is the number of elements we have in the btree. As
we update many indexes (7 at least), that mean we will keep 7xLog(N)
pages, each page being at least 4Ko (and this can be more for entries.
- if someone does not close a cursor, or even worse, if an external
client does not close a NamingEnumeration, then we will quickly get an
OOM on the server, or the cache will just block, forbidding any
modification to occur.
I'm really concerned by the second point.
Anyway, as a side effort, I have started to work on some B+Tree MVCC
project (mavibot labs), because I wanted to play around some new ideas,
after having read many papers about some new algorithms
(http://xxx.lanl.gov/abs/1103.4282) that may be very interesting to
implement. It's also my perception that such a project would be very
useful for many other projects than just Directory. Anwyay, this is
refreshing for me.
That's pretty much, I wanted to get you informed about what I was doing,
but keep in mind that I'm also 80% busy on side tasks. Sadly, I'm not
sure I can be full speed on ADS for at least one month or a bit more.
Thanks !
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
