[
https://issues.apache.org/jira/browse/LUCENE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533359#comment-13533359
]
Sivan Yogev commented on LUCENE-4258:
-------------------------------------
Let me start with the last question.
bq. Are stored fields now sparse? Meaning if I have a segment w/ many docs, and
I update stored fields on one doc, in that tiny stacked segments will the
stored fields files also be tiny?
In such case you will get the equivalent of a segment with multiple docs with
only one of them containing stored fields. I assume the impls of stored fields
handle these cases well and you will indeed get tiny stored fields.
Regarding the API - I made some cleanup, and removed also
Operation.ADD_DOCUMENT. Now there is only one way to perform each operation,
and updateFields only allows you to add or replace fields given a term.
bq. This means you cannot reuse fields, you have to be careful with
pre-tokenized fields (can't reuse the TokenStream), etc.
This is referred in the Javadoc of updateFields, let me know if there's a
better way to address it.
As for the heavier questions. NRT support should be considered separately, but
the guideline I followed was to keep things as closely as possible to the way
deletions are handled. Therefore, we need to add to SegmentReader a field named
liveUpdates - an equivalent to liveDocs. I already put a TODO for this
(SegmentReader line 131), implementing it won't be simple...
The performance tradeoff you are rightfully concerned about should be handled
through merging. Once you merge an updated segment all updates are "cleaned",
and the new segment has no performance issues. Apps that perform updates should
make sure (through MergePolicy) to avoid reader-side updates as much as
possible.
> Incremental Field Updates through Stacked Segments
> --------------------------------------------------
>
> Key: LUCENE-4258
> URL: https://issues.apache.org/jira/browse/LUCENE-4258
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
> Reporter: Sivan Yogev
> Attachments: IncrementalFieldUpdates.odp,
> LUCENE-4258-API-changes.patch, LUCENE-4258.r1410593.patch,
> LUCENE-4258.r1412262.patch, LUCENE-4258.r1416438.patch,
> LUCENE-4258.r1416617.patch
>
> Original Estimate: 2,520h
> Remaining Estimate: 2,520h
>
> Shai and I would like to start working on the proposal to Incremental Field
> Updates outlined here (http://markmail.org/message/zhrdxxpfk6qvdaex).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
