While I'm not an expert on the subject matter, I can't help myself but chime in and share my thoughts on this... =)
During the course of our project on analyzing Derby's SMP scalability properties, we've investigated Derby's locking and latching pretty thoroughly. We've followed this thread with interest as it's very much related to our ongoing work - and moving latches out of the data structure locks are managed in is one of the suggestions for improving scalability we have noted in our report. Anyway, one thing I've found while working on Derby is that the separation between interfaces and implementations can be a great advantage - adding the capability to use Java 1.4's NIO interfaces in RAFContainer was simplified a lot when we could just overload a few methods with a 1.4 specific class. Relating this to BasePage and latching, I believe that an implementation restricted to Java 1.3/1.4's concurrency control abilities will be rather disadvantaged compared to ReentrantLock and it's friends living in java.util.concurrent.locks in Java 1.5+. Now, I realise that I may sound like a broken record now - for DERBY-801 I jumped on the "new fancy features!" bandwagon, and I'm doing it again now. However, what I'm trying to convey this time isn't a strong desire to make something cool with the latest features - but to leave an avenue open for that at a later stage. This will be a bit harder (and probably a lot messier ;) if all the latching code is integrated into the BasePage code. At the very least I'd suggest factoring out latch functionality (state, synchronization, waiting) into a separate class. The LockFactory interface could (maybe with some modifications?) still be the factory interface for latches, even if they are not necessarily managed and stored in the big lock hash table. Giving the Lock factory responsibility for picking the appropriate latch implementation would not only save the BasePage implementation from containing a lot of latch logic, but it could also permit us to have, say, an implementation like the current one where deadlocks involving latches and locks can be debugged together for debugging work, a lightweight pure-java exclusive latch for Java 1.4 and possibly a shared/exclusive latch using ReentrantReadWriteLocks from Java 1.5. Now, I'm just mentioning the last one because we've observed significant wait times caused by contended latch waits on the root and second-level blocks in a B-tree index. Adding shared latching for B-tree traversing code could possibly be one way to alleviate this problem. Of course great care must be taken when working in this part of Derby and more thought should go into this before a decision on how to handle this is made, as Mike has pointed out it is easy to make a mistake here that causes silent corruption and data loss. (Not to mention that different latching implementations on different JVMs and debug/non-debug builds could lead to some very, uhm, "interesting" Heisenbugs... =) Anyway, I'm a newbie at this, but those are my two cents, and I'm just throwing my thoughts out here in the hope that someone finds a use for them. To reiterate - my point is that if we keep this architecturally clean, we can reap some benefits right away and keep the avenue open for more cool stuff later. My ideas for what could be "cool stuff" are not necessarily good ideas. =) Thanks, -- Anders Morken My opinions may have changed, but not the fact that I am right!
