On 5/3/2018 12:55 PM, Satya Marivada wrote: > We have a solr (6.3.0) index which is being re-indexed every night, it > takes about 6-7 hours for the indexing to complete. During the time of > re-indexing, the index becomes flaky and would serve inconsistent count of > documents 70,000 at times and 80,000 at times. After the indexing is > completed, it serves the consistent and right number of documents that it > has indexed from the database. Any suggestions on this.
Initial guess is that there are commits being fired before the whole indexing process is complete. If you're running in cloud mode, there could be other things going on. > Also solr writes to the same location as current index during re-indexing. > Could this be the cause of concern? When you use an existing index as the write location for a re-index, you must be very careful to ensure that you do not ever send any commit requests before the entire indexing process is complete. The autoCommit config in solrconfig.xml must have openSearcher set to false, and autoSoftCommit must not be active. That way, all queries sent before the process completes will be handled by the index that existed before the indexing process started. A commit when the process is done will send new queries to the new state of the index. An alternate idea would be to index the replacement index into a different core/collection, and then swap the indexes. In SolrCloud mode, the swap would be accomplished using the Collection Alias feature. Thanks, Shawn
