[
https://issues.apache.org/jira/browse/OAK-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-10048:
----------------------------
Description:
DocumentStoreIndexerBase#buildFlatFileStore below outputs the first
FlatFileStore and its path which is incorrect if the FlatFileSplitter is
triggered to split files based on config and index definitions.
Also, the buildFlatFileStoreList method always passes false to split but shoud
instead use the configuration {{IndexerConfiguration.parallelIndexEnabled()}}
to try splitting based on this configuration.
{code:java}
public FlatFileStore buildFlatFileStore() throws IOException,
CommitFailedException {
NodeState checkpointedState =
indexerSupport.retrieveNodeStateForCheckpoint();
Set<String> preferredPathElements = new HashSet<>();
Set<IndexDefinition> indexDefinitions = getIndexDefinitions();
for (IndexDefinition indexDf : indexDefinitions) {
preferredPathElements.addAll(indexDf.getRelativeNodeNames());
}
Predicate<String> predicate = s ->
indexDefinitions.stream().anyMatch(indexDef ->
indexDef.getPathFilter().filter(s) != PathFilter.Result.EXCLUDE);
FlatFileStore flatFileStore = buildFlatFileStoreList(checkpointedState,
null, predicate, preferredPathElements, false, indexDefinitions).get(0);
log.info("FlatFileStore built at {}. To use this flatFileStore in a
reindex step, set System Property-{} with value {}",
flatFileStore.getFlatFileStorePath(),
OAK_INDEXER_SORTED_FILE_PATH, flatFileStore.getFlatFileStorePath());
return flatFileStore;
}
{code}
was:
DocumentStoreIndexerBase#buildFlatFileStore below outputs the first
FlatFileStore and its path which is incorrect if the FlatFileSplitter is
triggered to split files based on config and index definitions.
{code:java}
public FlatFileStore buildFlatFileStore() throws IOException,
CommitFailedException {
NodeState checkpointedState =
indexerSupport.retrieveNodeStateForCheckpoint();
Set<String> preferredPathElements = new HashSet<>();
Set<IndexDefinition> indexDefinitions = getIndexDefinitions();
for (IndexDefinition indexDf : indexDefinitions) {
preferredPathElements.addAll(indexDf.getRelativeNodeNames());
}
Predicate<String> predicate = s ->
indexDefinitions.stream().anyMatch(indexDef ->
indexDef.getPathFilter().filter(s) != PathFilter.Result.EXCLUDE);
FlatFileStore flatFileStore = buildFlatFileStoreList(checkpointedState,
null, predicate, preferredPathElements, false, indexDefinitions).get(0);
log.info("FlatFileStore built at {}. To use this flatFileStore in a
reindex step, set System Property-{} with value {}",
flatFileStore.getFlatFileStorePath(),
OAK_INDEXER_SORTED_FILE_PATH, flatFileStore.getFlatFileStorePath());
return flatFileStore;
}
{code}
> DocumentStoreIndexerBase#buildFlatFileStore outputs the wrong path when
> FlatFileSplitter used
> ---------------------------------------------------------------------------------------------
>
> Key: OAK-10048
> URL: https://issues.apache.org/jira/browse/OAK-10048
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-run
> Reporter: Amit Jain
> Priority: Major
>
> DocumentStoreIndexerBase#buildFlatFileStore below outputs the first
> FlatFileStore and its path which is incorrect if the FlatFileSplitter is
> triggered to split files based on config and index definitions.
> Also, the buildFlatFileStoreList method always passes false to split but
> shoud instead use the configuration
> {{IndexerConfiguration.parallelIndexEnabled()}} to try splitting based on
> this configuration.
> {code:java}
> public FlatFileStore buildFlatFileStore() throws IOException,
> CommitFailedException {
> NodeState checkpointedState =
> indexerSupport.retrieveNodeStateForCheckpoint();
> Set<String> preferredPathElements = new HashSet<>();
> Set<IndexDefinition> indexDefinitions = getIndexDefinitions();
> for (IndexDefinition indexDf : indexDefinitions) {
> preferredPathElements.addAll(indexDf.getRelativeNodeNames());
> }
> Predicate<String> predicate = s ->
> indexDefinitions.stream().anyMatch(indexDef ->
> indexDef.getPathFilter().filter(s) != PathFilter.Result.EXCLUDE);
> FlatFileStore flatFileStore =
> buildFlatFileStoreList(checkpointedState, null, predicate,
> preferredPathElements, false, indexDefinitions).get(0);
> log.info("FlatFileStore built at {}. To use this flatFileStore in a
> reindex step, set System Property-{} with value {}",
> flatFileStore.getFlatFileStorePath(),
> OAK_INDEXER_SORTED_FILE_PATH, flatFileStore.getFlatFileStorePath());
> return flatFileStore;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
