Author: adulceanu
Date: Tue Oct 22 15:56:36 2019
New Revision: 1868765
URL: http://svn.apache.org/viewvc?rev=1868765&view=rev
Log:
OAK-8703 - Configure the maximum number of revisions to be checked by
consistency check
Contribution by Ieran Draghiciu
Modified:
jackrabbit/oak/trunk/oak-doc/src/site/markdown/nodestore/segment/overview.md
jackrabbit/oak/trunk/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CheckCommand.java
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tooling/ConsistencyChecker.java
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/tool/Check.java
Modified:
jackrabbit/oak/trunk/oak-doc/src/site/markdown/nodestore/segment/overview.md
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-doc/src/site/markdown/nodestore/segment/overview.md?rev=1868765&r1=1868764&r2=1868765&view=diff
==============================================================================
---
jackrabbit/oak/trunk/oak-doc/src/site/markdown/nodestore/segment/overview.md
(original)
+++
jackrabbit/oak/trunk/oak-doc/src/site/markdown/nodestore/segment/overview.md
Tue Oct 22 15:56:36 2019
@@ -727,13 +727,11 @@ This tool is the counterpart of `backup`
### <a name="check"/> Check
```
-java -jar oak-run.jar check PATH [--mmap] [--journal JOURNAL] [--notify SECS]
[--bin] [--head] [--checkpoints all | cp1[,cp2,..,cpn]] [--filter
PATH1[,PATH2,..,PATHn]] [--io-stats]
+java -jar oak-run.jar check PATH [--mmap] [--journal JOURNAL] [--notify SECS]
[--bin] [--last <REV_COUNT>] [--head] [--checkpoints all | cp1[,cp2,..,cpn]]
[--filter PATH1[,PATH2,..,PATHn]] [--io-stats]
```
The `check` tool inspects an existing Segment Store at `PATH` for eventual
inconsistencies.
-The algorithm implemented by this tool traverses every revision in the
journal, from the most recent to the oldest.
-For every revision, the actual nodes and properties are traversed, verifying
that every piece of data is reachable and undamaged. Moreover, if `--head` and
`--checkpoints` options are used, the scope of the traversal can be limited to
head state and/or a subset of checkpoints.
-A deep scan of the content tree, traversing every node and every property will
be performed by default. The default scope includes head state and all
checkpoints.
+The algorithm implemented by this tool traverses every revision in the
journal, from the most recent to the oldest, stopping at the first consistent
occurence. The actual nodes and properties are traversed, verifying that every
piece of data is reachable and undamaged. If `--last` option is present, the
tool will start with the most recent revision and will go back in the history
at most `<REV_COUNT>` revisions. Moreover, if `--head` and `--checkpoints`
options are used, the scope of the traversal can be limited to head state
and/or a subset of checkpoints. A deep scan of the content tree, traversing
every node and every property will be performed by default. The default scope
includes head state and all checkpoints.
The optional `--mmap [Boolean]` argument can be used to control the file
access mode. Set
to `true` for memory mapped access and `false` for file access (default is
`true`).
@@ -749,6 +747,8 @@ If the `--bin` option is specified, the
If not specified, the binary properties will not be traversed.
The `--bin` option has no effect on binary properties stored in an external
Blob Store.
+The optional `--last [Integer]` argument can be used to control the maximum
number of revisions to be verified (default is `1`).
+
If the `--head` option is specified, the tool will scan **only** the head
state, ignoring any available checkpoints.
If the `--checkpoints` option is specified, the tool will scan **only** the
specified checkpoints, ignoring the head state. At least one argument is
expected with this option; multiple arguments need to be comma-separated.
Modified:
jackrabbit/oak/trunk/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CheckCommand.java
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CheckCommand.java?rev=1868765&r1=1868764&r2=1868765&view=diff
==============================================================================
---
jackrabbit/oak/trunk/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CheckCommand.java
(original)
+++
jackrabbit/oak/trunk/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CheckCommand.java
Tue Oct 22 15:56:36 2019
@@ -45,6 +45,9 @@ class CheckCommand implements Command {
.withRequiredArg()
.ofType(Long.class)
.defaultsTo(Long.MAX_VALUE);
+ OptionSpec<Integer> last = parser.accepts("last", "define the number
of revisions to be checked (default: 1)")
+ .withOptionalArg()
+ .ofType(Integer.class);
OptionSpec<?> bin = parser.accepts("bin", "read the content of binary
properties");
OptionSpec<String> filter = parser.accepts("filter", "comma separated
content paths to be checked")
.withRequiredArg()
@@ -87,6 +90,10 @@ class CheckCommand implements Command {
builder.withJournal(journal.value(options));
}
+ if (options.has(last)) {
+ builder.withRevisionsCount(options.valueOf(last) != null ?
last.value(options) : 1);
+ }
+
System.exit(builder.build().run());
}
Modified:
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tooling/ConsistencyChecker.java
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tooling/ConsistencyChecker.java?rev=1868765&r1=1868764&r2=1868765&view=diff
==============================================================================
---
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tooling/ConsistencyChecker.java
(original)
+++
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tooling/ConsistencyChecker.java
Tue Oct 22 15:56:36 2019
@@ -345,12 +345,13 @@ public class ConsistencyChecker {
boolean head,
Set<String> checkpoints,
Set<String> paths,
- boolean binaries
+ boolean binaries,
+ Integer revisionsCount
) {
List<PathToCheck> headPaths = new ArrayList<>();
Map<String, List<PathToCheck>> checkpointPaths = new HashMap<>();
- int revisionCount = 0;
+ int checkedRevisionsCount = 0;
for (String path : paths) {
if (head) {
@@ -373,7 +374,7 @@ public class ConsistencyChecker {
String revision = journalEntry.getRevision();
try {
- revisionCount++;
+ checkedRevisionsCount++;
store.setRevision(revision);
onCheckRevision(revision);
@@ -398,6 +399,12 @@ public class ConsistencyChecker {
if (allPathsConsistent(headPaths, checkpointPaths)) {
break;
}
+
+ // limit the number of revisions to be checked
+
+ if (checkedRevisionsCount == revisionsCount) {
+ break;
+ }
} catch (IllegalArgumentException | SegmentNotFoundException e) {
onCheckRevisionError(revision, e);
}
@@ -405,7 +412,7 @@ public class ConsistencyChecker {
ConsistencyCheckResult result = new ConsistencyCheckResult();
- result.checkedRevisionsCount = revisionCount;
+ result.checkedRevisionsCount = checkedRevisionsCount;
result.overallRevision = newRevisionOrNull(lastValidJournalEntry);
for (PathToCheck path : headPaths) {
Modified:
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/tool/Check.java
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/tool/Check.java?rev=1868765&r1=1868764&r2=1868765&view=diff
==============================================================================
---
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/tool/Check.java
(original)
+++
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/tool/Check.java
Tue Oct 22 15:56:36 2019
@@ -78,6 +78,8 @@ public class Check {
private boolean checkHead;
+ private Integer revisionsCount;
+
private Set<String> checkpoints;
private Set<String> filterPaths;
@@ -171,6 +173,17 @@ public class Check {
}
/**
+ * Instruct the command to check only the last {@code revisionsCount}
revisions.
+ * This parameter is not required and defaults to {@code 1}.
+ * @param revisionsCount number of revisions to check.
+ * @return this builder.
+ */
+ public Builder withRevisionsCount(Integer revisionsCount){
+ this.revisionsCount = revisionsCount;
+ return this;
+ }
+
+ /**
* Instruct the command to check specified checkpoints.
* This parameter is not required and defaults to "/checkpoints",
* i.e. will check all checkpoints when not explicitly overridden.
@@ -298,6 +311,8 @@ public class Check {
private final boolean checkHead;
+ private final Integer revisionsCount;
+
private final Set<String> requestedCheckpoints;
private final Set<String> filterPaths;
@@ -333,6 +348,7 @@ public class Check {
this.out = builder.outWriter;
this.err = builder.errWriter;
this.journal = journalPath(builder.path, builder.journal);
+ this.revisionsCount = revisionsToCheckCount(builder.revisionsCount);
}
private static File journalPath(File segmentStore, File journal) {
@@ -342,6 +358,10 @@ public class Check {
return journal;
}
+ private static Integer revisionsToCheckCount(Integer revisionsCount) {
+ return revisionsCount != null ? revisionsCount : Integer.MAX_VALUE;
+ }
+
public int run() {
StatisticsIOMonitor ioMonitor = new StatisticsIOMonitor();
@@ -390,7 +410,8 @@ public class Check {
checkHead,
checkpoints,
filterPaths,
- checkBinaries
+ checkBinaries,
+ revisionsCount
);
print("\nSearched through {0} revisions and {1} checkpoints",
result.getCheckedRevisionsCount(), checkpoints.size());
