I was kind of wondering what magic had been done to achieve multiple writing to
the index file :-)
BTW, wouldn't it be possible to have seperate segments per thread? Set up the
index with a minimum (desired?) segment count, and write each individually?
Is there any organization in the segments? Or can adjacent data be found in
different segments?
I seem to remember that the new stuff gets committed to its own segment until
some sort of 'consolidate' command takes place.
Dennis Gearon
Signature Warning
EARTH has a Right To Life,
otherwise we all die.
Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Thu, 9/23/10, Jan Høydahl / Cominvent jan@cominvent.com wrote:
From: Jan Høydahl / Cominvent jan@cominvent.com
Subject: Re: is indexing single-threaded?
To: solr-user@lucene.apache.org
Date: Thursday, September 23, 2010, 1:42 AM
SolrJ threads speeds up feeding
throughput. The building the index is still single threaded
(per core), isn't it? Don't know about analysis. But you
cannot have two threads write to the same file...
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
On 23. sep. 2010, at 08.01, Ryan McKinley wrote:
Multiple threads work well.
If you are using solrj, check the StreamingSolrServer
for an
implementation that will keep X number of threads
busy.
Your mileage will very, but in general I find a
reasonable thread
count is ~ (number of cores)+1
On Wed, Sep 22, 2010 at 5:52 AM, Andy angelf...@yahoo.com
wrote:
Does Solr index data in a single thread or can
data be indexed concurrently in multiple threads?
Thanks
Andy
