[
https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072619#comment-16072619
]
Chetan Mehrotra commented on OAK-5192:
--------------------------------------
bq. how can I set this up in a unit test like
https://github.com/tteofili/jackrabbit-oak/blob/oak-5192/oak-lucene/src/test/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneSegmentStatsTest.java
?
[~teofili] You would need to configure a FileDataStore with minRecordLength set
to 4000. A rough approach is below which you would need to adapt
{noformat}
NodeStore nodeStore;
try {
fileStore = FileStoreBuilder.fileStoreBuilder(DIRECTORY)
.withStatisticsProvider(new
DefaultStatisticsProvider(scheduledExecutorService))
.withBlobStore(createBlobStore())
.build();
nodeStore = SegmentNodeStoreBuilders.builder(fileStore).build();
} catch (IOException | InvalidFileStoreVersionException e) {
throw new RuntimeException(e);
}
BlobStore createBlobStore(){
FileDataStore fds = new OakFileDataStore();
fds.setPath(new File("path/tods"));
fds.setMinRecordLength(4000);
fds.init(null);
return new DataStoreBlobStore(fds);
}
{noformat}
> Reduce Lucene related growth of repository size
> -----------------------------------------------
>
> Key: OAK-5192
> URL: https://issues.apache.org/jira/browse/OAK-5192
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene, segment-tar
> Reporter: Michael Dürig
> Assignee: Tommaso Teofili
> Labels: perfomance, scalability
> Fix For: 1.8, 1.7.8
>
> Attachments: added-bytes-zoom.png, binSize100.txt, binSize16384.txt,
> binSizeTotal.txt, diff.txt.zip, nonBinSizeTotal.txt, OAK-5192.0.patch, Screen
> Shot 2017-07-03 at 16.50.00.png
>
>
> I observed Lucene indexing contributing to up to 99% of repository growth.
> While the size of the index itself is well inside reasonable bounds, the
> overall turnover of data being written and removed again can be as much as
> 99%.
> In the case of the TarMK this negatively impacts overall system performance
> due to fast growing number of tar files / segments, bad locality of
> reference, cache misses/thrashing when looking up segments and vastly
> prolonged garbage collection cycles.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
