Hi,
I am starting to use solr, now I need to index a rather large amount of data,
it seems
that calling solr to pass data through HTTP is rather inefficient, I am think
still call
lucene API directly for bulk index but to use solr for search, is this design
OK?
Thanks very much for helps,
Hello!
If you use Java (and I think you do, because you mention Lucene) you
should take a look at StreamingUpdateSolrServer. It not only allows
you to send data in batches, but also index using multiple threads.
--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
On 7/26/2012 7:34 AM, Rafał Kuć wrote:
If you use Java (and I think you do, because you mention Lucene) you
should take a look at StreamingUpdateSolrServer. It not only allows
you to send data in batches, but also index using multiple threads.
A caveat to what Rafał said:
The streaming object
Thanks very much, both your and Rafal's advice are very helpful!
-Original Message-
From: Shawn Heisey [mailto:s...@elyograg.org]
Sent: Thursday, July 26, 2012 8:47 AM
To: solr-user@lucene.apache.org
Subject: Re: Bulk indexing data into solr
On 7/26/2012 7:34 AM, Rafał Kuć wrote
Right in time, guys. https://issues.apache.org/jira/browse/SOLR-3585
Here is server side update processing fork. It does the best for halting
processing on exception occurs. Plug this UpdateProcessor, specify number
of threads. Then submit lazy iterator into StreamingUpdateServer at client
side.
Coming back to your original question. I'm puzzled a little.
It's not clear where you wanna call Lucene API directly from.
if you mean that you has standalone indexer, which write index files. Then
it stops and these files become available for Solr Process it will work.
Sharing index between
@lucene.apache.org
Subject: Re: Bulk indexing data into solr
Coming back to your original question. I'm puzzled a little.
It's not clear where you wanna call Lucene API directly from.
if you mean that you has standalone indexer, which write index files. Then
it stops and these files become available
AM
To: solr-user@lucene.apache.org
Subject: Re: Bulk indexing data into solr
Coming back to your original question. I'm puzzled a little.
It's not clear where you wanna call Lucene API directly from.
if you mean that you has standalone indexer, which write index files. Then
it stops
-Original Message-
From: Mikhail Khludnev [mailto:mkhlud...@griddynamics.com]
Sent: Thursday, July 26, 2012 12:46 PM
To: solr-user@lucene.apache.org
Subject: Re: Bulk indexing data into solr
IIRC about a two month ago problem with such scheme discussed here, but I
can remember exact
