hey, On Thu, Jul 19, 2012 at 7:41 PM, Simon McDuff <smcd...@hotmail.com> wrote: > > Thank you for your answer! > > I read all your blogs! It is always interesting!
for details see: http://www.searchworkings.org/blog/-/blogs/gimme-all-resources-you-have-i-can-use-them!/ and http://www.searchworkings.org/blog/-/blogs/lucene-indexing-gains-concurrency/ > > My understanding is probably incorrect ... > I observed that if you have only one thread that addDocument, it will not > spawn another thread for flushing, it uses the main thread. every indexing thread can hit a flush. if you only have one thread you will not make progress adding docs while flushing. IW will not create new threads for flushing. > In this case, my main thread is locked. Correct ? > > The concurrent flushing will ONLY work when I have many threads adding > documents ? (In that case I will need to put a ringbuffer in front) that is basically correct. You can frequently call commit / or pull a reader from the IW in a different thread before you ram buffer fills up so that flushing happens in a different thread. That could work pretty well if you don't have many deletes to be applied. (if you have many deletes then pull a reader without applying deletes. simon > > Do I understand correctly ? Did I miss something ? > > Simon > >> From: luc...@mikemccandless.com >> Date: Thu, 19 Jul 2012 13:02:42 -0400 >> Subject: Re: Flushing Thread >> To: java-user@lucene.apache.org >> >> This has already been fixed on Lucene 4.0 (we now have fully >> concurrent flushing), eg see: >> >> >> http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> On Thu, Jul 19, 2012 at 12:54 PM, Simon McDuff <smcd...@hotmail.com> wrote: >> > >> > I see some behavior at the moment when I'm flushing and would like to know >> > if I can change that. >> > >> > One main thread is inserting, when it flushes, it blocks. >> > During that time my main thread is blocking. Instead of blocking, Could >> > it spawn another thread to do that ? >> > >> > Basically, would like to have one main thread adding document to my >> > index, if a flushing needs to occur, spawn another threads but it should >> > never lock the main threads. Is it possible ? >> > >> > Is the only solution is to have many threads indexing the data ? >> > In that case Is it true to say ONLY one of them will be busy while the >> > other is flushing ? (I do understand that if my flushing is taking two >> > much time, they will both flush... :-)) >> > >> > Thank you! >> > >> > Simon >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
