--Original Message- From: Peter Keegan
> Sent: Thursday, November 6, 2014 3:21 PM
> To: java-user
> Subject: Exceptions during batch indexing
>
>
> How are folks handling Solr exceptions that occur during batch indexing?
> Solr (4.6) stops parsing the docs stream when an er
Oops... you sent this to the wrong list - this is the Lucene user list, send
it to the Solr user list.
-- Jack Krupansky
-Original Message-
From: Peter Keegan
Sent: Thursday, November 6, 2014 3:21 PM
To: java-user
Subject: Exceptions during batch indexing
How are folks handling Solr
How are folks handling Solr exceptions that occur during batch indexing?
Solr (4.6) stops parsing the docs stream when an error occurs (e.g. a doc
with a missing mandatory field), and stops indexing. The bad document is
not identified, so it would be hard for the client to recover by skipping
over
ks again,
Paul
-Original Message-
From: java-user-return-45439-paul.b.murdoch=saic@lucene.apache.org
[mailto:java-user-return-45439-paul.b.murdoch=saic@lucene.apache.org
] On Behalf Of Erick Erickson
Sent: Monday, March 15, 2010 12:45 PM
To: java-user@lucene.apache.org
Subject: Re: Batch
-Original Message-
> From: java-user-return-45433-paul.b.murdoch=saic@lucene.apache.org
> [mailto:java-user-return-45433-paul.b.murdoch=saic@lucene.apache.org
> ] On Behalf Of Mark Miller
> Sent: Monday, March 15, 2010 10:48 AM
> To: java-user@lucene.apache.org
> Subject:
:48 AM
To: java-user@lucene.apache.org
Subject: Re: Batch Indexing - best practice?
On 03/15/2010 10:41 AM, Murdoch, Paul wrote:
Hi,
I'm using Lucene 2.9.2. Currently, when creating my index, I'm
calling
indexWriter.addDocument(doc) for each Document I want to ind
:java-user-return-45433-paul.b.murdoch=saic@lucene.apache.org
] On Behalf Of Mark Miller
Sent: Monday, March 15, 2010 10:48 AM
To: java-user@lucene.apache.org
Subject: Re: Batch Indexing - best practice?
On 03/15/2010 10:41 AM, Murdoch, Paul wrote:
> Hi,
>
>
>
> I'm using
ble. I have the merge factor set to
> 1000. Do you have any suggestions for batch indexing? Is there
> something like indexWriter.addDocuments(Document[] docs) in the API?
>
>
>
> Thanks.
>
> Paul
>
>
>
>
-
econds. I'd like to try and speed this upunless 90
seconds for 500 Documents is reasonable. I have the merge factor set to
1000. Do you have any suggestions for batch indexing? Is there
something like indexWriter.addDocuments(Document[] docs) in the API?
Thanks.
Paul
You shoul
unless 90
seconds for 500 Documents is reasonable. I have the merge factor set to
1000. Do you have any suggestions for batch indexing? Is there
something like indexWriter.addDocuments(Document[] docs) in the API?
Thanks.
Paul
ot figure out where it
>> went
>> wrong, and how come this strange behavior: 520 documents but 0 terms.
>>
>> I would be very grateful if anyone could advise. THanks very much.
>>
>> yanyan
>>
>>
>> --
>> View this message in conte
g, and how come this strange behavior: 520 documents but 0 terms.
I would be very grateful if anyone could advise. THanks very much.
yanyan
--
View this message in context:
http://www.nabble.com/bad-index-by-batch-indexing-tp18862037p18862037.html
Sent from the Lucene - Java Users mailing list a
d be very grateful if anyone could advise. THanks very much.
>
> yanyan
>
>
> --
> View this message in context:
> http://www.nabble.com/bad-index-by-batch-indexing-tp18862037p18862037.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
&g
very grateful if anyone could advise. THanks very much.
yanyan
--
View this message in context:
http://www.nabble.com/bad-index-by-batch-indexing-tp18862037p18862037.html
Sent from the Lucene - Java Users mailing list archive at Nabble
For some intermediate period of time when indexing, the document are
buffered in RAM. There is a complex interplay between several of the
parameters to an IndexWriter that govern how many documents are
indexed in RAM before being flushed to the FSDirectory.
Of course, if you specify a RAMdirector
Sorry , ask a question.
You say FSDirectory is RADdirectory as least until it flushes.
I cannot understand your means . May you please teach me what it means?
FSDirectory stored in filesystem , and RADdirectory stored in RAM.
MergeFactor and MaxBufferedDocs settings are limited and controlled
MAIL PROTECTED]> wrote:
Thanks Erik , so FSDirectory seems better option than RAMDirectory ? Also
I
think O.S can cache files in which case FSDirectory may not be bad , your
thoughts ?
-Original Message-
From: Erick Erickson [mailto:[EMAIL PROTECTED]
Sent: Sunday, April 29, 2007 7:07
@lucene.apache.org
Subject: Re: batch indexing
As I understand it, FSDirectory *is* RAMdirectory, at least until
it flushes. There have been several discussions of this,
search the mail archive for things like MergeFactor, MaxBufferedDocs
and the like. You'll find quite a bit of information about how
As I understand it, FSDirectory *is* RAMdirectory, at least until
it flushes. There have been several discussions of this,
search the mail archive for things like MergeFactor, MaxBufferedDocs
and the like. You'll find quite a bit of information about how these
parameters interact.
Particularly, s
I am trying to index a huge documents on batches . Batch size is
parameterized to the application say X docs , that means it will hold X no.
of
Docs in the RAM before I flush to file system using
IndexWriter.addIndexes(Directory[]) method
My question is :
Do I need to set mergefactor ?
Hey Eric,
I think you want:
fsWriter.addIndexes(Directory[] {ramDir});
to be:
fsWriter.addIndexes(new Directory[]{ramDir});
JAMES
--- zheng <[EMAIL PROTECTED]> wrote:
> I am a novice in lucene. I write some code to do
> batch indexing using
> RAMDirectory according to the c
I am a novice in lucene. I write some code to do batch indexing using
RAMDirectory according to the code provided in lucene in action, which is
something like FSDirectory fsDir = FSDirectory.getDirectory("/tmp/index",
true);
RAMDirectory ramDir = new RAMDirectory();
IndexWrite
22 matches
Mail list logo
