Re: Stress/scalability testing Lucene

[Doug Cutting]

Otis Gospodnetic wrote:

Reding and writing at the same time is okay.  Only one thread can
modify the index at a time.

Almost. Only one process can modify it at a time, other processes will be prevented by the write.lock file. Multiple threads can modify an index simultaneously. The bulk of the work of the updates will be serialized by synchronization in IndexWriter, but the analysis of the text into tokens is parallelizeable.

Doug

--- [EMAIL PROTECTED] wrote:

Ah, for some reason i thought none of the Lucene methods were thread
safe,
or is this only in the case of reading and writing at the same time? I
thought I read this in the FAQ.

Roy.

-----Original Message-----
From: Doug Cutting [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 20, 2002 5:04 PM
To: Lucene Users List
Subject: Re: Stress/scalability testing Lucene


* Replies will be sent through Spamex to
[EMAIL PROTECTED]
* For additional info click -> http://www.spamex.com/i/?v=886513

Justin Greene wrote:

We created a thread pool to read and parse the email
messages.  10 threads seems to be the magic number here for us.  We

then

created a queue of messages to be indexed onto which we push the

parsed

messages and have a single thread adding messages to the index.

IndexWriter.addDocument(Document) is thread safe, so you don't need a

separate indexing thread. So long as your analyzer is thread safe,
you can index each messages in the thread that parses it, for even
greater parallelism.

Doug


--
To unsubscribe, e-mail:
<mailto:[EMAIL PROTECTED]>
For additional commands, e-mail:
<mailto:[EMAIL PROTECTED]>


This email and any attachments are confidential and may be legally privileged. No confidentiality or privilege is waived or lost by any transmission in error. If you are not the intended recipient you are hereby notified that any use, printing, copying or disclosure is strictly prohibited. Please delete this email and any attachments, without printing, copying, forwarding or saving them and notify the sender immediately by reply e-mail. Zurich Capital Markets and its affiliates reserve the right to monitor all e-mail communications through its networks. Unless otherwise stated, any pricing information in this e-mail is indicative only, is subject to change and does not constitute an offer to enter into any transaction at such price and any terms in relation to any proposed transaction are indicative only and subject to express final confirmation.

__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - Let the expert host your site
http://webhosting.yahoo.com

--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>