fabriziofortino commented on a change in pull request #383:
URL: https://github.com/apache/jackrabbit-oak/pull/383#discussion_r735765174
##########
File path:
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/DocumentStoreIndexerBase.java
##########
@@ -71,7 +71,7 @@
protected List<NodeStateIndexerProvider> indexerProviders;
protected final IndexerSupport indexerSupport;
private final Set<String> indexerPaths = new HashSet<>();
- private static final int MAX_DOWNLOAD_RETRIES =
Integer.parseInt(System.getProperty("oak.indexer.maxDownloadRetries", "3"));
+ private static final int MAX_DOWNLOAD_ATTEMPTS =
Integer.parseInt(System.getProperty("oak.indexer.maxDownloadRetries", "5")) + 1;
public DocumentStoreIndexerBase(IndexHelper indexHelper, IndexerSupport
indexerSupport) throws IOException {
Review comment:
this should not throw IOE
##########
File path:
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/DocumentStoreIndexerBase.java
##########
@@ -167,10 +162,20 @@ private FlatFileStore buildFlatFileStore(NodeState
checkpointedState, CompositeI
} catch (CompositeException e) {
e.logAllExceptions("Underlying throwable caught during
download", log);
log.info("Could not build flat file store. Execution count {}.
Retries left {}. Time elapsed {}",
- executionCount, MAX_DOWNLOAD_RETRIES - executionCount,
flatFileStoreWatch);
+ executionCount, MAX_DOWNLOAD_ATTEMPTS -
executionCount, flatFileStoreWatch);
lastException = e;
previousDownloadDirs.add(builder.getFlatFileStoreDir());
}
+ if (executionCount < MAX_DOWNLOAD_ATTEMPTS) {
+ try {
+ log.info("Waiting for {} millis before retrying",
backOffTimeInMillis);
+ Thread.sleep(backOffTimeInMillis);
+ backOffTimeInMillis *= 2;
+ } catch (InterruptedException e) {
+ log.error("Interrupted while waiting before retrying
download ", e);
+ e.printStackTrace();
Review comment:
this should not be needed since the instruction above should print the
stack trace
##########
File path:
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/MultithreadedTraverseWithSortStrategy.java
##########
@@ -52,6 +52,8 @@
import java.util.stream.Stream;
import static com.google.common.base.Charsets.UTF_8;
+import static
org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileNodeStoreBuilder.DEFAULT_NUMBER_OF_DATA_DUMP_THREADS;
+import static
org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileNodeStoreBuilder.PROP_THREAD_POOL_SIZE;
import static
org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreUtils.createWriter;
import static
org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreUtils.getSortedStoreFileName;
Review comment:
The link in the javadoc below should be
`FlatFileNodeStoreBuilder#PROP_THREAD_POOL_SIZE`
##########
File path:
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/TraverseAndSortTask.java
##########
@@ -71,10 +73,9 @@
private long memoryUsed;
private final File sortWorkDir;
private final List<File> sortedFiles = new ArrayList<>();
- private final ArrayList<NodeStateHolder> entryBatch = new ArrayList<>();
+ private final LinkedList<NodeStateHolder> entryBatch = new LinkedList<>();
private NodeStateEntry lastSavedNodeStateEntry;
private final String taskID;
- private final AtomicBoolean dumpData = new AtomicBoolean(false);
Review comment:
remove import of `AtomicBoolean`
##########
File path:
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/FlatFileNodeStoreBuilder.java
##########
@@ -54,6 +54,17 @@
*/
Review comment:
remote import of `LastModifiedRange`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
