Author: chetanm
Date: Fri Jun 14 12:46:39 2013
New Revision: 1493053
URL: http://svn.apache.org/r1493053
Log:
OAK-619 Lock-free MongoMK implementation
Documentation Work
-- Updated README with details around background job and some open topics
-- Changed variable names for better readability of code
Modified:
jackrabbit/oak/trunk/oak-mongomk/README.md
jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
Modified: jackrabbit/oak/trunk/oak-mongomk/README.md
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-mongomk/README.md?rev=1493053&r1=1493052&r2=1493053&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-mongomk/README.md (original)
+++ jackrabbit/oak/trunk/oak-mongomk/README.md Fri Jun 14 12:46:39 2013
@@ -1,139 +1,167 @@
-Oak MongoMK
-===========
-
-This module contains a `MicroKernel` implementation using MongoDB to persist
-content.
-
-Content Model
--------------
-
-The `MongoMK` stores each node in a separate MongoDB document and updates to
-a node are stored by adding new revision/value pairs to the document. This way
-the previous state of a node is preserved and can still be retrieved by a
-session looking a given snapshot (revision) of the repository.
-
-The basic MongoDB document of a node in Oak looks like this:
-
- {
- "_id" : "1:/node",
- "_deleted" : {
- "r13f3875b5d1-0-1" : "false"
- },
- "_lastRev" : {
- "1" : "r13f3875b5d1-0-1"
- },
- "_modified" : NumberLong(274208361),
- "_revisions" : {
- "r13f3875b5d1-0-1" : "true"
- }
- }
-
-All fields in the above document are metadata and are not exposed through the
-Oak API.
-
-The `_id` field is used as the primary key and consists of a combination of the
-depth of the path and the path itself. This is an optimization to align sibling
-keys in the index.
-
-The `_deleted` sub-document contains the revision this node was created in. In
-the above example the root node was created in revision `r13f3875b5d1-0-1`. If
-the node is later deleted, the `_deleted` sub-document will get a new field
with
-the revision the node was deleted in.
-
-The sub-document `_lastRev` contains the last revision written to this node by
-each cluster node. In the above example the MongoMK cluster node with id `1`
-modified the node the last time in revision `r13f3875b5d1-0-1`, when it created
-the node.
-
-The `_modified` field contains a low-resolution timestamp when the node was
last
-modified. The time resolution is five seconds.
-
-Finally, the `_revision` sub-document contains commit information about changes
-marked with a revision. E.g. the single entry in the above document tells us
-that everything marked with revision `r13f3875b5d1-0-1` is committed and
-therefore valid.
-
-Adding a property `prop` with value `foo` to the node in a next step will
-result in the following document:
-
- {
- "_deleted" : {
- "r13f3875b5d1-0-1" : "false"
- },
- "_id" : "1:/node",
- "_lastRev" : {
- "1" : "r13f38818ab6-0-1"
- },
- "_modified" : NumberLong(274208516),
- "_revisions" : {
- "r13f3875b5d1-0-1" : "true",
- "r13f38818ab6-0-1" : "true"
- },
- "prop" : {
- "r13f38818ab6-0-1" : "\"foo\""
- }
- }
-
-Now the document contains a new sub-document with the name of the new property.
-The value of the property is annotated with the revision the property was set.
-With each successful commit to this node, a new field is added to the
-`_revision` sub-document. Similarly the `_lastRev` sub-document and `_modified`
-field are updated.
-
-After the node is deleted the document looks like this:
-
- {
- "_deleted" : {
- "r13f3875b5d1-0-1" : "false",
- "r13f38835063-2-1" : "true"
- },
- "_id" : "1:/node",
- "_lastRev" : {
- "1" : "r13f38835063-2-1"
- },
- "_modified" : NumberLong(274208539),
- "_revisions" : {
- "r13f3875b5d1-0-1" : "true",
- "r13f38818ab6-0-1" : "true",
- "r13f38835063-2-1" : "true"
- },
- "prop" : {
- "r13f38818ab6-0-1" : "\"foo\""
- }
- }
-
-The `_deleted` sub-document now contains a `r13f38835063-2-1` field marking the
-node as deleted in this revision.
-
-Reading the node in previous revisions is still possible, even if it is now
-marked as deleted as of revision `r13f38835063-2-1`.
-
-Revision Model
---------------
-
-* Explain revision, cluster node id, etc.
-* Explain branches
-
-
-
-License
--------
-
-(see the top-level [LICENSE.txt](../LICENSE.txt) for full license details)
-
-Collective work: Copyright 2013 The Apache Software Foundation.
-
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+Oak MongoMK
+===========
+
+This module contains a `MicroKernel` implementation using MongoDB to persist
+content.
+
+Content Model
+-------------
+
+The `MongoMK` stores each node in a separate MongoDB document and updates to
+a node are stored by adding new revision/value pairs to the document. This way
+the previous state of a node is preserved and can still be retrieved by a
+session looking a given snapshot (revision) of the repository.
+
+The basic MongoDB document of a node in Oak looks like this:
+
+ {
+ "_id" : "1:/node",
+ "_deleted" : {
+ "r13f3875b5d1-0-1" : "false"
+ },
+ "_lastRev" : {
+ "1" : "r13f3875b5d1-0-1"
+ },
+ "_commitRoot": {
+ "r13f3875b5d1-0-1": 0
+ },
+ "_modified" : NumberLong(274208361),
+ "_revisions" : {
+ "r13f3875b5d1-0-1" : "true"
+ }
+ }
+
+All fields in the above document are metadata and are not exposed through the
+Oak API.
+
+The `_id` field is used as the primary key and consists of a combination of the
+depth of the path and the path itself. This is an optimization to align sibling
+keys in the index.
+
+The `_deleted` sub-document contains the revision this node was created in. In
+the above example the root node was created in revision `r13f3875b5d1-0-1`. If
+the node is later deleted, the `_deleted` sub-document will get a new field
with
+the revision the node was deleted in.
+
+The sub-document `_lastRev` contains the last revision written to this node by
+each cluster node. In the above example the MongoMK cluster node with id `1`
+modified the node the last time in revision `r13f3875b5d1-0-1`, when it created
+the node.
+
+The `_modified` field contains a low-resolution timestamp when the node was
last
+modified. The time resolution is five seconds.
+
+The sub-document `_commitRoot` contains commit root depth for the commit in
which
+the node was created against the revision.
+
+Finally, the `_revision` sub-document contains commit information about changes
+marked with a revision. E.g. the single entry in the above document tells us
+that everything marked with revision `r13f3875b5d1-0-1` is committed and
+therefore valid. In case the change is done in a branch then the value would
be the
+base revision. It is only added for thode nodes which happen to be the commit
root
+for any give commit.
+
+Adding a property `prop` with value `foo` to the node in a next step will
+result in the following document:
+
+ {
+ "_deleted" : {
+ "r13f3875b5d1-0-1" : "false"
+ },
+ "_id" : "1:/node",
+ "_lastRev" : {
+ "1" : "r13f38818ab6-0-1"
+ },
+ "_modified" : NumberLong(274208516),
+ "_revisions" : {
+ "r13f3875b5d1-0-1" : "true",
+ "r13f38818ab6-0-1" : "true"
+ },
+ "prop" : {
+ "r13f38818ab6-0-1" : "\"foo\""
+ }
+ }
+
+Now the document contains a new sub-document with the name of the new property.
+The value of the property is annotated with the revision the property was set.
+With each successful commit to this node, a new field is added to the
+`_revision` sub-document. Similarly the `_lastRev` sub-document and `_modified`
+field are updated.
+
+After the node is deleted the document looks like this:
+
+ {
+ "_deleted" : {
+ "r13f3875b5d1-0-1" : "false",
+ "r13f38835063-2-1" : "true"
+ },
+ "_id" : "1:/node",
+ "_lastRev" : {
+ "1" : "r13f38835063-2-1"
+ },
+ "_modified" : NumberLong(274208539),
+ "_revisions" : {
+ "r13f3875b5d1-0-1" : "true",
+ "r13f38818ab6-0-1" : "true",
+ "r13f38835063-2-1" : "true"
+ },
+ "prop" : {
+ "r13f38818ab6-0-1" : "\"foo\""
+ }
+ }
+
+The `_deleted` sub-document now contains a `r13f38835063-2-1` field marking the
+node as deleted in this revision.
+
+Reading the node in previous revisions is still possible, even if it is now
+marked as deleted as of revision `r13f38835063-2-1`.
+
+Revision Model
+--------------
+
+* Explain revision, cluster node id, etc.
+* Explain branches
+
+Background Operations
+---------------------
+Each MongoMK instance connecting to same database in Mongo server performs
certain backgroun task
+
+### Renew Cluster Id Lease
+
+### Background Writes
+
+While performing commits there are certain nodes which are modified but do not
become part
+of commit. For example when a node under /a/b/c is updated then the `_lastRev`
property also
+needs to be updated to the commit revision. Such changes are accumulated and
flushed periodically
+through a asynch job
+
+### Background Reads
+
+
+Pending Topics
+--------------
+
+### Conflict Detection and Handling
+
+
+License
+-------
+
+(see the top-level [LICENSE.txt](../LICENSE.txt) for full license details)
+
+Collective work: Copyright 2013 The Apache Software Foundation.
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
Modified:
jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java?rev=1493053&r1=1493052&r2=1493053&view=diff
==============================================================================
---
jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
(original)
+++
jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
Fri Jun 14 12:46:39 2013
@@ -60,7 +60,12 @@ public class Commit {
private final Revision revision;
private HashMap<String, UpdateOp> operations = new HashMap<String,
UpdateOp>();
private JsopWriter diff = new JsopStream();
- private HashSet<String> changedNodes = new HashSet<String>();
+
+ /**
+ * List of all node paths which have been modified in this commit. In
addition to the nodes
+ * which are actually changed it also contains there parent node paths
+ */
+ private HashSet<String> modifiedNodes = new HashSet<String>();
private HashSet<String> addedNodes = new HashSet<String>();
private HashSet<String> removedNodes = new HashSet<String>();
@@ -152,18 +157,18 @@ public class Commit {
/**
* Apply the changes to the document store (to update MongoDB).
*
- * @param baseRevision the base revision of this commit. Currently only
+ * @param baseBranchRevision the base revision of this commit. Currently
only
* used for branch commits.
*/
- void applyToDocumentStore(Revision baseRevision) {
+ void applyToDocumentStore(Revision baseBranchRevision) {
// the value in _revisions.<revision> property of the commit root node
// regular commits use "true", which makes the commit visible to
// other readers. branch commits use the base revision to indicate
// the visibility of the commit
- String commitValue = baseRevision != null ? baseRevision.toString() :
"true";
+ String commitValue = baseBranchRevision != null ?
baseBranchRevision.toString() : "true";
DocumentStore store = mk.getDocumentStore();
String commitRootPath = null;
- if (baseRevision != null) {
+ if (baseBranchRevision != null) {
// branch commits always use root node as commit root
commitRootPath = "/";
}
@@ -172,6 +177,8 @@ public class Commit {
// operations are added to this list before they are executed,
// so that all operations can be rolled back if there is a conflict
ArrayList<UpdateOp> opLog = new ArrayList<UpdateOp>();
+
+ //Compute the commit root
for (String p : operations.keySet()) {
markChanged(p);
if (commitRootPath == null) {
@@ -472,7 +479,7 @@ public class Commit {
}
list.add(p);
}
- for (String path : changedNodes) {
+ for (String path : modifiedNodes) {
ArrayList<String> added = new ArrayList<String>();
ArrayList<String> removed = new ArrayList<String>();
ArrayList<String> changed = nodesWithChangedChildren.get(path);
@@ -508,7 +515,7 @@ public class Commit {
throw new IllegalArgumentException("path: " + path);
}
while (true) {
- changedNodes.add(path);
+ modifiedNodes.add(path);
if (PathUtils.denotesRoot(path)) {
break;
}
