Hi Otis The only way to continually use the same IndexReader would be to use a stateful session bean (frowned upon by J2EE Container writers), and then we'd see the same problem cross-machine - remember that the indexes that we use are stored once, not replicated across the four servers.
We'll come up with a fix ourselves. I thought that it might be a good candidate for Lucene 2 as the FSDirectory code is horrible and non-J2EE compliant. Is there any place for making suggestions for future development tasks? I've seen the Lucene 2 whiteboard and wondered how the items on the list were derived. Cheers Pete Lewis ----- Original Message ----- From: "Otis Gospodnetic" <[EMAIL PROTECTED]> To: "Lucene Developers List" <[EMAIL PROTECTED]> Sent: Tuesday, September 14, 2004 12:11 PM Subject: Re: Lock handling and Lucene 1.9 / 2.0 > It sounds like this is the source of your problem. Reuse those > IndexReaders or IndexSearchers and you'll avoid the waits you are > talking about, as Christoph already pointed out. How you implement the > logic for caching IndexReaders/Searchers is up to you. > > Otis > P.S. > It sounds like you took the time to figure out how some of the Lucene > internals work. The best way to persuade -dev list subscribers that > doing something one way is better than doing it the current way is by > providing a clean diff against CVS HEAD, attached to an entry in > Bugzilla. There are now several active Lucene developers and > contributors who will look at your suggestion and apply the patch, if > it improves the code. > > > > --- Pete Lewis <[EMAIL PROTECTED]> wrote: > > > Hi Christoph > > > > We are in a cluster running under Bea Weblogic. > > > > We have a static API to the search component from the portlets. > > > > We therefore need to open an indexreader per request. > > > > Cheers > > Pete > > > > ----- Original Message ----- > > From: "Christoph Goller" <[EMAIL PROTECTED]> > > To: "Lucene Developers List" <[EMAIL PROTECTED]> > > Sent: Tuesday, September 14, 2004 10:55 AM > > Subject: Re: Lock handling and Lucene 1.9 / 2.0 > > > > > > > Pete Lewis wrote: > > > > Hi Christoph > > > > > > > > If we stand back a second and ask why we have commit locks when > > searching? > > > > > > > > The answer comes down to handling the FSDirectory - where the > > methods > > used > > > > are not j2ee compliant. > > > > > > > > We could open another can of worms and say why does the > > indexreader > > delete - > > > > but I won't go into that argument again here..... > > > > > > > > The bottom line is that we need the ability to search without > > waiting on > > a > > > > commit lock. > > > > > > You have this ability already !!!!!!!!!!!!!!!!!!!!!!!!! > > > > > > The FSDirectory is where the problems lie. We could hack the > > > > code to make it work for our particular application; however what > > I've > > been > > > > trying to get across is the need to have a method that will give > > users > > the > > > > capability to just search (not delete) without waiting upon the > > commit > > lock, > > > > that will be j2ee compliant, and that will be appropriate > > clustered > > > > implementations - and that this should be a candidate for Lucene > > 1.9 / > > 2.0. > > > > > > > > You say that it shouldn't take long to wait. A 1 sec spin lock > > per > > index > > > > per process is an eternity when trying to scale for potentially > > thousands of > > > > users. > > > > > > I have to admit that I am not an expert in j2ee compliancy. But I > > would > > like > > > to learn about it. If a database (I consder Lucene as a database) > > really > > has > > > to be initialized for every read-access, than there is a problem > > with j2ee > > > compliancy. I cannot believe that this is really true. > > > > > > LET ME STATE AGAIN: You should not open a new IndexReader for every > > > search/query. If you do so you definitely have a performance > > problem > > > independently from synchronization!!!!!!!!! Opening an IndexReader > > is > > > much more expensive than any individual query/search. > > > > > > You need a commit.lock for opening an IndexReader not because > > IndexReader > > > has delete functionality. You need it because if there is some > > process > > > writing to your index, your index may be in an inconsistent state. > > An > > existing > > > commit.lock indicates such an inconsistent state. Therefore, every > > writer > > needs > > > a commit.lock while committing, and every reader needs a > > commit.lock while > > > opening an index. > > > > > > Christoph > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
