[
https://issues.apache.org/jira/browse/SOLR-12999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058905#comment-18058905
]
Jason Gerlowski commented on SOLR-12999:
----------------------------------------
Houston and I found some reference-counting and resource-closing issues in this
IndexFetcher codepath; see SOLR-18121 for more details. The nature of the bugs
suggests that we have 0 tests for this feature. (Which makes sense to be fair
- it's hard for an automated test to exercise a codepath that triggers only
when disks are sufficiently full.)
Additionally - looking back on it 5 years on - this feature just feels kindof
risky overall. If anything happens to the leader _after_ the replica has
deleted its local files, we could end up with data loss.
Do we think this feature still makes sense in Solr today? If not - any
objections to my removing it? If we would like to keep it around - what do you
all think of gating it behind a system property, etc. so users have to opt in
to this behavior?
(FYI [~dsmiley]...saw you initially reported this, so you might have the most
context and most cogent argument on why this is still needed)
> Index replication could delete segments first
> ---------------------------------------------
>
> Key: SOLR-12999
> URL: https://issues.apache.org/jira/browse/SOLR-12999
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Reporter: David Smiley
> Assignee: Noble Paul
> Priority: Major
> Fix For: 8.1
>
> Attachments: SOLR-12999.patch, SOLR-12999.patch
>
>
> Index replication could optionally delete files that it knows will not be
> needed _first_. This would reduce disk capacity requirements of Solr, and it
> would reduce some disk fragmentation when space get tight.
> Solr (IndexFetcher) already grabs the remote file list, and it could see
> which files it has locally, then delete the others. Today it asks Lucene to
> {{deleteUnusedFiles}} at the end. This new mode would probably only be
> useful if there is no SolrIndexSearcher open, since it would prevent the
> removal of files.
> The motivating scenario is a SolrCloud replica that is going into full
> recovery. It ought to not be fielding searches. The code changes would not
> depend on SolrCloud though.
> This option would have some danger the user should be aware of. If the
> replication fails, leaving the local files incomplete/corrupt, the only
> recourse is to try full replication again. You can't just give up and field
> queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
