[jira] [Updated] (SOLR-12085) IndexFetcher does not honor SolrDeletionPolicy

2018-03-22 Thread Karishma Agrawal (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karishma Agrawal updated SOLR-12085:

Affects Version/s: 7.1
   7.2

> IndexFetcher does not honor SolrDeletionPolicy
> --
>
> Key: SOLR-12085
> URL: https://issues.apache.org/jira/browse/SOLR-12085
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
>Affects Versions: 5.4.1, 7.1, 7.2
>Reporter: Karishma Agrawal
>Priority: Major
>
> Index Fetcher is not in sync with Solr Deletion Policy.
> If we set the SolrDeletionPolicy to retain more than 1 commit - i.e. 
> maxCommitsToKeep > 1. Then slaves get stuck in alternate full replication. 
> This happens because within IndexFetcher, there is a check for unusedFiles - 
> hasUnusedFile, which takes IndexDirectory and latest commit as parameters. If 
> there are files unrelated to latest commit in the index dir then this method 
> returns true, and SolrDeletionPolicy is invoked. However there are no pending 
> files to delete according to the index deletion policy since that is aware of 
> other valid commits, and no action is taken. Once all the retries are 
> exhausted, index fetcher sets fullreplication to true.  
> Possible solutions: 
>  # remove the check for hasUnusedFile completely. We invoke 
> IndexWriter#deleteUnusedFiles and move on.
>  # Add another method in IndexFileDeleter (Lucene) which returns number of 
> pending deleted files. We can replace the IndexFetcher#hasUnusedFile method 
> with this.
>  # Keep track of unused files in IndexDeletionPolicyWrapper.     
> A variation of this bug was previously noted in 
> https://issues.apache.org/jira/browse/SOLR-9278    



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-12085) IndexFetcher does not honor SolrDeletionPolicy

2018-03-13 Thread Karishma Agrawal (JIRA)
Karishma Agrawal created SOLR-12085:
---

 Summary: IndexFetcher does not honor SolrDeletionPolicy
 Key: SOLR-12085
 URL: https://issues.apache.org/jira/browse/SOLR-12085
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: replication (java)
Affects Versions: 5.4.1
Reporter: Karishma Agrawal


Index Fetcher is not in sync with Solr Deletion Policy.

If we set the SolrDeletionPolicy to retain more than 1 commit - i.e. 
maxCommitsToKeep > 1. Then slaves get stuck in alternate full replication. This 
happens because within IndexFetcher, there is a check for unusedFiles - 
hasUnusedFile, which takes IndexDirectory and latest commit as parameters. If 
there are files unrelated to latest commit in the index dir then this method 
returns true, and SolrDeletionPolicy is invoked. However there are no pending 
files to delete according to the index deletion policy since that is aware of 
other valid commits, and no action is taken. Once all the retries are 
exhausted, index fetcher sets fullreplication to true.  

Possible solutions: 
 # remove the check for hasUnusedFile completely. We invoke 
IndexWriter#deleteUnusedFiles and move on.
 # Add another method in IndexFileDeleter (Lucene) which returns number of 
pending deleted files. We can replace the IndexFetcher#hasUnusedFile method 
with this.
 # Keep track of unused files in IndexDeletionPolicyWrapper.     

A variation of this bug was previously noted in 
https://issues.apache.org/jira/browse/SOLR-9278    



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org