[
https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959051#comment-15959051
]
Andrei Dulceanu edited comment on OAK-4122 at 4/6/17 3:28 PM:
--------------------------------------------------------------
[~mduerig]
bq. I think there is no need to use SegmentNodeState.fastEquals directly (and
consequently make that method public). Instead just compare the node states
through equals, which should then delegate to fastEquals (please double check
on your end).
If I do it this way for {{Commit#hasChanges}}, [uncommenting this line
|https://github.com/dulceanu/jackrabbit-oak/commit/dd1ec86cd4d3539180af1c51e844cdef31f5a090#diff-49cb01594447d1b595aa62c536d6e803R101],
I always end up with these tests failing:
{code}
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.41 sec <<<
FAILURE! - in
org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest
blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest)
Time elapsed: 0.082 sec <<< ERROR!
java.io.IOException: java.lang.NullPointerException
Caused by: java.lang.NullPointerException
Running
org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.163 sec <<<
FAILURE! - in
org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest)
Time elapsed: 0.062 sec <<< ERROR!
java.io.IOException: java.lang.NullPointerException
Caused by: java.lang.NullPointerException
{code}
However, the same change works really well
[here|https://github.com/dulceanu/jackrabbit-oak/blob/features/scheduler/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/scheduler/Commit.java#L67].
-Do you think this needs further investigation?- Opened OAK-6051 to track this.
If not, I will apply the changes in the branch in trunk, resolve this issue and
open follow up issues for the remaining tasks.
was (Author: dulceanu):
[~mduerig]
bq. I think there is no need to use SegmentNodeState.fastEquals directly (and
consequently make that method public). Instead just compare the node states
through equals, which should then delegate to fastEquals (please double check
on your end).
If I do it this way for {{Commit#hasChanges}}, [uncommenting this line
|https://github.com/dulceanu/jackrabbit-oak/commit/dd1ec86cd4d3539180af1c51e844cdef31f5a090#diff-49cb01594447d1b595aa62c536d6e803R101],
I always end up with these tests failing:
{code}
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.41 sec <<<
FAILURE! - in
org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest
blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.ExternalToExternalMigrationTest)
Time elapsed: 0.082 sec <<< ERROR!
java.io.IOException: java.lang.NullPointerException
Caused by: java.lang.NullPointerException
Running
org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.163 sec <<<
FAILURE! - in
org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest
blobsExistsOnTheNewBlobStore(org.apache.jackrabbit.oak.segment.migration.SegmentToExternalMigrationTest)
Time elapsed: 0.062 sec <<< ERROR!
java.io.IOException: java.lang.NullPointerException
Caused by: java.lang.NullPointerException
{code}
However, the same change works really well
[here|https://github.com/dulceanu/jackrabbit-oak/blob/features/scheduler/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/scheduler/Commit.java#L67].
Do you think this needs further investigation? If not, I will apply the
changes in the branch in trunk, resolve this issue and open follow up issues
for the remaining tasks.
> Replace the commit semaphore in the segment node store with a scheduler
> -----------------------------------------------------------------------
>
> Key: OAK-4122
> URL: https://issues.apache.org/jira/browse/OAK-4122
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
> Reporter: Michael Dürig
> Assignee: Andrei Dulceanu
> Labels: operations, performance, scalability, throughput
> Fix For: 1.8, 1.7.3
>
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent
> commits thus relying on the scheduling algorithm of that implementation and
> ultimately of the JVM for in what order commits are processed.
> I think it would be beneficial to replace that semaphore with an explicit
> queue of pending commit. This would allow us to implement a proper scheduler
> optimising for e.g. minimal system load, maximal throughput or minimal
> latency etc. A scheduler could e.g. give precedence to big commits and order
> commits along the order of its base revisions, which would decrease the
> amount of work to be done in rebasing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
