Simon Willnauer commented on LUCENE-8198:

> There are some advantages of doing it this way far as performance goes I 
>think. For example, take the extreme case where a user never "hard"-deletes at 
>all. They'd still always get bulk merge optimizations from the stored fields 
>writer, etc.

that is a good one. Yet, I think the case where we never had hard deletes are 
unlikely true for this usecase. It only applies if there are deletes at all at 
least that is what I was focusing on. 

> Well the overhead should be... 1 bit per document unless something is wrong? 
>So a user would add something like a "soft live" field and then just update it 
>to a 0. I think its easy to filter on such fields in searches already.

there is additonal memory overhead when indexing which can be not massive for 
deletes / DV updates but still. I also try to understand how you can use 
updateable docvalues to do that in an atomic fashion. it must be somehow part 
of updateDocument which I wonder if it's the right way to go. 



> Add ability to persist deletes across merges
> --------------------------------------------
>                 Key: LUCENE-8198
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8198
>             Project: Lucene - Core
>          Issue Type: Improvement
>    Affects Versions: 7.3, master (8.0)
>            Reporter: Simon Willnauer
>            Assignee: Simon Willnauer
>            Priority: Major
>         Attachments: LUCENE-8198.patch
> This allows conditionally persist deletes on a per document basis to prevent 
> them from being merged away. This expert feature is useful to maintain 
> history of documents in the index where otherwise a duplicate storage 
> mechanism would be needed. For instance features like CouchDBs changes API 
> can be build on top of persistent deletes. While using persistent deletes has 
> a considerably small overhead at merge time or when deletes applied to fully 
> deleted segments, there is no impact if persistent deletes are unused.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to