[
https://issues.apache.org/jira/browse/OAK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15476613#comment-15476613
]
Chetan Mehrotra commented on OAK-4412:
--------------------------------------
Thanks [~tmueller] for the feedback!
bq. What is the easiest way to have a "collapsed" patch (compare trunk against
latest version of your patch)?
Use [this
url|https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-4412.diff]
which provides a single unified diff
bq. Because the patch is large, it's hard to review. But reviewing intermediate
steps is probably even more work
Ack that patch is large but thats just due to scope of feature work and its
being done in a branch. I still try to break down the commit with proper
message to try to simplify future review. Also tried to provide an overview of
the design in previous comment to aid in review. Going forward if you have any
suggestion on such large feature are implemented then I can keep those in mind
bq. Possible impact on concurrency by adding "synchronized" for createReader
createReader would only be called by single thread which would be refreshing
the reader as per ReaderRefreshPolicy implementation. So should be fine.
bq. It looks like unrelated issues are touched, for example lazy init of
"FacetsConfig facetsConfig". Would be better done in separate issues.
For most cases I created separate issue (see various linked issues for this
issue). Just that this one I did as part of this patch. Can you try to separate
it and rebase my branch
bq. To reduce impact on existing code, it would be good if the new feature is
"modular", so if it is not used, the risk is low
Yes its possible. This whole new code path would only be used if index
definition is marked with "nrt" or "sync". Otherwise not much difference
between old code path and new code. So if such flag are not used
* LuceneIndexEditorProvider - Whole new if block here would not get used and
old code path would be taken
* IndexNode would have null nrtIndex and there also whole new logic would not
get triggered
bq. //TODO [hybrid] ... not sure if all of them are resolved?
Some are left and I would prefer to address them once code is merged to trunk
as they do not block things and more of good to have. Addressing them now would
further delay and increase the patch size
bq. Basic code conventions not follows (as always with code you wrote), for
example sometimes missing space before "{"
Aah .. would try to improve there. Though it helps you later to differentiate
what part of code is new ... right ;)
> Lucene hybrid index
> -------------------
>
> Key: OAK-4412
> URL: https://issues.apache.org/jira/browse/OAK-4412
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: lucene
> Reporter: Tomek Rękawek
> Assignee: Chetan Mehrotra
> Fix For: 1.6
>
> Attachments: OAK-4412-v1.diff, OAK-4412.patch, hybrid-benchmark.sh,
> hybrid-result-v1.txt
>
>
> When running Oak in a cluster, each write operation is expensive. After
> performing some stress-tests with a geo-distributed Mongo cluster, we've
> found out that updating property indexes is a large part of the overall
> traffic.
> The asynchronous index would be an answer here (as the index update won't be
> made in the client request thread), but the AEM requires the updates to be
> visible immediately in order to work properly.
> The idea here is to enhance the existing asynchronous Lucene index with a
> synchronous, locally-stored counterpart that will persist only the data since
> the last Lucene background reindexing job.
> The new index can be stored in memory or (if necessary) in MMAPed local
> files. Once the "main" Lucene index is being updated, the local index will be
> purged.
> Queries will use an union of results from the {{lucene}} and
> {{lucene-memory}} indexes.
> The {{lucene-memory}} index, as a local stored entity, will be updated using
> an observer, so it'll get both local and remote changes.
> The original idea has been suggested by [~chetanm] in the discussion for the
> OAK-4233.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
