[
https://issues.apache.org/jira/browse/OAK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-2749:
---------------------------------
Attachment: OAK-2749-v3.diff
attaching proposed patch. still WIP as there are parts missing.
I'm proposing changes in the Oak class that will allow for passing in the
(name, frequency) of the async indexing tasks. this will open the door for
tweaking the number of async tasks as well as reducing the frequency for some
that are too intensive or could allow for more lag. I'm keeping backwards
compatibility here by changing the 'withAsyncIndexing()' method a bit.
The online lane addition would happen via a JMX call which will be responsible
for setting up the checkpoints correctly. The input would be _the list of paths
to the index needing update_ (simplest way I could think of, without having to
do a full traversal to identify all existing index defs from the repository)
and of course the _new task name_. Here I added 2 versions of the method, not
being sure how accessible would be a jmx call that takes a _Set_ as an input.
The remaining problem is how to boot a new indexing task while being online.
I'm looking at the _RepositoryManagementMBean_ as a candidate, but this is
still very much a TODO at this point.
one issue that needs consideration after applying this patch is the checkpoints
'rm-unreferenced' cleanup call from oak-run. it will no longer work properly,
and as we don't have a name convention yet, it will be a bit tricky to fix. One
name convention would be that the async task names should start with 'async',
but for now it's just a loose idea.
cc. [~chetanm], [~edivad] feedback appreciated!
> Provide a "different lane" for slow indexers in async indexing
> --------------------------------------------------------------
>
> Key: OAK-2749
> URL: https://issues.apache.org/jira/browse/OAK-2749
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
> Reporter: Davide Giannella
> Assignee: Davide Giannella
> Fix For: 1.3.0
>
> Attachments: OAK-2749-rc1.diff, OAK-2749-rc2.diff, OAK-2749-v3.diff
>
>
> In case of big repositories, asynchronous index like Lucene Property,
> could lag behind as slow indexes, for example Full Text, are taken
> care in the same thread pool.
> Provide a separate thread pool in which such indexes could be
> registered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
