Hi guys,
Using solr 3.6.1 and the following settings, I am trying to run without
optimizes. I used to optimize nightly, but sometimes the optimize took a very
long time to complete and slowed down our indexing. We are continuously
indexing our new or changed data all day and night. After a few days running
without an optimize, the index size has nearly doubled and maxdocs is nearly
twice the size of numdocs. I understand deletes should be expunged on merges,
but even after trying lots of different settings for our merge policy it seems
this growth is somewhat unbounded. I have tried sending an optimize with
numSegments = 2 which is a lot lighter weight then a regular optimize and that
does bring the number down but not by too much. Does anyone have any ideas for
better settings for my merge policy that would help? Here is my current index
snapshot too:
Location: /var/LucidWorks/lucidworks/solr/10000/data/index
Size: 25.05 GB (when the index is optimized it is around 15.5 GB)
searcherName : Searcher@6c3a3517 main
caching : true
numDocs : 16852155
maxDoc : 24512617
reader :
SolrIndexReader{this=6e3b4ec8,r=ReadOnlyDirectoryReader@6e3b4ec8,refCnt=1,segments=61}
<mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
<int name="maxMergeAtOnce">35</int>
<int name="segmentsPerTier">35</int>
<int name="maxMergeAtOnceExplicit">105</int>
<double name="maxMergedSegmentMB">6144.0</double>
<double name="reclaimDeletesWeight">8.0</double>
</mergePolicy>
<mergeScheduler
class="org.apache.lucene.index.ConcurrentMergeScheduler">
<int name="maxMergeCount">20</int>
<int name="maxThreadCount">3</int>
</mergeScheduler>
Thanks,
Robert (Robi) Petersen
Senior Software Engineer
Search Department
(formerly Buy.com)
85 enterprise, suite 100
aliso viejo, ca 92656
tel 949.389.2000 x5465
fax 949.448.5415
