[
https://issues.apache.org/jira/browse/LUCENE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14121314#comment-14121314
]
Michael McCandless commented on LUCENE-5895:
--------------------------------------------
Hmm, it depends on how merging works b/w primary & replica? E.g. we could
merge only on primary and copy merged segments out. Yes, this is added
bandwidth, but then merging is also CPU / IO intensive, so saving that work for
the replicas may net/net be worthwhile. And copying out a merged segment to a
replica can be a lowish priority thing, e.g. it need not gate NRT reopen time
(just like how IW's merged segment warmer doesn't either).
But yeah if the replicas do their own merging, somehow with a merge policy
that's guaranteed to precisely match what the primary does (despite e.g.
ConcurrentMergeScheduler causing merges to complete in different orders), then
we would need to somehow set the IDs of the merged segments on the replicas to
match the corresponding merged segments on the primaries. Or, perhaps ... we
compute the ID of a merged segment by hashing the IDs of the N segments that
were merged ... hmm risky because if there were different deleted docs then the
IDs should differ, so maybe we hash on that too ... tricky.
> Add per-segment and per-commit id to help replication
> -----------------------------------------------------
>
> Key: LUCENE-5895
> URL: https://issues.apache.org/jira/browse/LUCENE-5895
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 5.0, 4.11
>
> Attachments: LUCENE-5895.patch, LUCENE-5895.patch
>
>
> It would be useful if Lucene recorded a unique id for each segment written
> and each commit point. This way, file-based replicators can use this to
> "know" whether the segment/commit they are looking at on a source machine and
> dest machine are in fact that same.
> I know this would have been very useful when I was playing with NRT
> replication (LUCENE-5438).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
