Yes, GC gets pretty bad even with only 8G of RAM. I also tried using RAM
disk and use SimpleFSDirectory, which performs well and keeps the java heap
small, but with this many indices, it actually keeps hundreds of thousands
files open. This is not really a Lucene question, but could that cause
prob
On Fri, Apr 26, 2013 at 5:04 PM, Aleksey wrote:
> Thanks for the response, Mike. Yes, I've come upon your blog before, it's
> very helpful.
>
> I tried bigger batches, it seems the highest throughput I can get is
> roughly 250 docs a second. From your blog, you updated your index at about
> 1MB pe
Thanks for the response, Mike. Yes, I've come upon your blog before, it's
very helpful.
I tried bigger batches, it seems the highest throughput I can get is
roughly 250 docs a second. From your blog, you updated your index at about
1MB per second, with 1K documents, which is 1000/s, but you had 24
Batching the updates really ought to improve overall throughput. Have you
tried with even bigger batches (100,1000 docs)?
But, how large is each update? Are you changing any IndexWriter settings,
e.g. ramBufferSizeMB.
Using threads should help too, at least a separate thread doing indexing
from
Hey guys,
I'm new to Lucene and I was trying to estimate how fast I can make updates
to the index and reopen it. The behavior I'm seeing seems odd.
I'm using Lucene4.2 and SearchManager instance that takes an index writer.
I make a loop where I update 1 document, then call maybeRefresh and acquir
