[
https://issues.apache.org/jira/browse/OAK-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18029793#comment-18029793
]
Julian Sedding commented on OAK-11934:
--------------------------------------
In order to configure preloading, a persistent cache needs to also be
configured. The code snippet below shows an example of how to configure the
{{FileStoreBuilder}} to enable preloading.
{code:java}
FileStore fileStore = FileStoreBuilder.fileStoreBuilder(folder)
.withPersistentCache(new PersistentDiskCache(...))
.withPersistentCachePreloading(PersistentCachePreloadingConfiguration.withConcurrency(32).withMaxPreloadDepth(6))
.build();
{code}
> segment preloading for PersistentCache
> --------------------------------------
>
> Key: OAK-11934
> URL: https://issues.apache.org/jira/browse/OAK-11934
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
> Reporter: Julian Sedding
> Assignee: Julian Sedding
> Priority: Major
> Fix For: 1.88.0
>
>
> Particularly for remote segment stores, IO can be a constraining factor.
> Processes like compaction, that traverse the repository, often alternate
> between processing segments and loading segments.
> IO could be parallelized by asynchronously preloading segments that are
> referenced by a newly loaded segment into a {{PersistentCache}}. I.e. if the
> "main" thread requests a segment from the cache, and the segment needs to be
> loaded from the persistence, then all segments referenced by the newly loaded
> segment are preloaded, and placed into the cache, asynchronously. When the
> "main" thread loads the next segment, it is likely already in the cache.
> Preloading could preload a configurable "depth" of references. Presumably,
> usually a depth of 1 or 2 strikes a good balance between preloading too
> aggressively and efficiently parallelizing IO.
> If preloading of references is only performed for newly loaded segments, the
> overhead of the preload mechanism should be minimal to non-existent while
> only cached segments are read.
> cc [~miroslav], [~nuno.santos]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
