Mark, I don't understand this; of course it is use case specific, I haven't seen any terrible behaviour with 8Gb... 32Mb is extremely small for Nutch-SOLR -like applications, but it is acceptable for Liferay-SOLR...
Please note also, I have some documents with same IDs updated many thousands times a day, and I believe (I hope) IndexWriter flushes "optimized" segment instead of thousands "delete" and single "insert" in many small (32Mb) files (especially with SOLR)... > Hmm - came out worse than it looked. Here is a better attempt: > > MergeFactor: 10 > > BUF DOCS/S > 32 37.40 > 80 39.91 > 120 40.74 > 512 38.25 > > Mark Miller wrote: > > Here is an example using the Lucene benchmark package. Indexing 64,000 > > wikipedia docs (sorry for the formatting): > > > > [java] ------------> Report sum by Prefix (MAddDocs) and Round (4 > > about 32 out of 256058) > > [java] Operation round mrg flush runCnt > > recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem > > [java] MAddDocs_8000 0 10 32.00MB 8 > > 8000 37.40 1,711.22 124,612,472 182,689,792 > > [java] MAddDocs_8000 - 1 10 80.00MB - - 8 - - - 8000 - > > - 39.91 - 1,603.76 - 266,716,128 - 469,925,888 > > [java] MAddDocs_8000 2 10 120.00MB 8 > > 8000 40.74 1,571.02 348,059,488 548,233,216 > > [java] MAddDocs_8000 - 3 10 512.00MB - - 8 - - - 8000 - > > - 38.25 - 1,673.05 - 746,087,808 - 926,089,216 > > > > After about 32-40, you don't gain much, and it starts decreasing once > > you start getting to high. 8GB is a terrible recommendation. > >
