Hi, There are a few scenarios where the optimistic locking approach used by the SegmentMK fails in practice:
1) A large batch operation while other smaller changes are being committed. 2) Lots of concurrent changes being committed against the same journal. In scenario 1 the large operation like an import can't complete itself since while it is rebasing itself and re-applying commit hooks other smaller operations have already updated the journal, triggering new rounds of rebasing and hook processing for the large operation until it bails out with the "System overloaded" exception. In scenario 2 the same "System overloaded" exception occurs once there are too many concurrent changes for the system to keep up with when using just a single journal. As noted by Marcel and others, this case comes up pretty quickly in a benchmark that explicitly tries to push the system to the limit. While in scenario 2 the "System overloaded" exception is a valid alternative to a potentially prolonged wait until the commit can go through, in scenario 1 it is clearly troublesome. Thus I'd like to address it, and the solution I have in mind actually works for both cases: When encountering a case where the optimistic locking mechanism can't push a commit through in say one second, instead of waiting for a longer while I'd have the SegmentMK fall back to pessimistic locking where it explicitly acquired a hard lock on the journal and does the rebase/hook processing one more time while holding that lock. This guarantees that all commits will go through eventually (unless there's a conflict or a validation failure), while keeping the benefits of optimistic locking for most cases. And even for scenario 1 the bulk of the commit has already been persisted when the pessimistic locking kicks in, so the critical section should still be much smaller than with Jackrabbit 2.x where the lock is held also while the change set is being persisted. BR, Jukka Zitting
