[
https://issues.apache.org/jira/browse/LUCENE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917173#action_12917173
]
Michael McCandless commented on LUCENE-2655:
--------------------------------------------
bq. Ok, I have been stuck/excited about not having to use/understand the
remap-docids method, because it's hard to debug. However I see what you're
saying, and why remap-docids exists. I'll push the DWP buffered deletes to the
flushed deletes.
I think we still must remap, at least on the pushed (deletesFlushed) deletes?
On the buffered deletes for the DWPT (deletesInRAM), I think we can make these
relative to the DWPT (ie start from 0), but on pushing them into flushed
deletes we re-base them?
bq. This large cost is from loading the terms index and deleted docs?
Yes. We don't (hopefully) load norms, field cache, etc.
bq. When those large segments are merged though, the IO cost is so substantial
that loading tii or del into RAM probably doesn't account for much of the
aggregate IO, they're probably in the noise?
Well, the applyDeletes method is sync'd, vs merging which is fully concurrent.
(Also, merging doesn't load the tii).
bq. Or are you referring to the NRT apply deletes flush, however that is on a
presumably pooled reader?
Right, it would be pooled for the NRT case, so this is only a (sizable) perf
problem for the non-nrt case.
bq. Or you're just saying that today we're applying deletes across the board to
all segments prior to a merge, regardless of whether or not they're even
involved in the merge? It seems like that is changeable?
Right! That's what we do today (apply deletes to all segs) whereas it's really
only necessary to apply them to the segments being merged. I opened
LUCENE-2680 to track this.
> Get deletes working in the realtime branch
> ------------------------------------------
>
> Key: LUCENE-2655
> URL: https://issues.apache.org/jira/browse/LUCENE-2655
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Jason Rutherglen
> Fix For: Realtime Branch
>
> Attachments: LUCENE-2655.patch
>
>
> Deletes don't work anymore, a patch here will fix this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
