Re: isCurrent says no, but contents still invisible
That's strange. Since you don't close the writer usually adding the doc would not modify the index (unless adding the doc triggered a merge). You may want to check that: 1. writer and reader really opened against the same path; 2. reader isCurrent state also before adding the doc and after re-opening; 3. searched terms vs. added terms - might not be related to concurrency at all. Finally perhaps post here the code so people can take a look. Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 12:45:08: I'm trying what should be the dumbest possible example of concurrency management with 2.0 in Java with an ordinary FSDirectory. I create an IndexWriter from a pathname, an IndexReader from the same pathname, and an IndexSearcher from the reader. I add one document. I call isCurrent() on the reader. It says, 'false'. So, I close the reader and the searcher, and I create a new reader and a new searcher. I search for the document, and I don't find it. I must be missing something simple. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: isCurrent says no, but contents still invisible
#2 is a possible issue. I stared at the code some more: The test case adds up to : Create all the objects. Add three docs. Add a fourth doc. Do a query aimed at the fourth doc. isCurrent() returns false. Close reader/searcher, open reader/searcher, numDocs() in the reader returns 3. Not 4. However, reading your message carefully, I realize that I'm probably fundamentally misguided. There's no flush() API on a writer, so, of course, the only possible way for a reader to see current contents is if the writer gets closed and reopened. I keep trying to cook up some scheme in which this is not true, but, with the stock classes, it now seems self-evident to me that it has to be true. I'll put in the requisite code, and slink away. -Original Message- From: Doron Cohen [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 4:07 PM To: java-user@lucene.apache.org Subject: Re: isCurrent says no, but contents still invisible That's strange. Since you don't close the writer usually adding the doc would not modify the index (unless adding the doc triggered a merge). You may want to check that: 1. writer and reader really opened against the same path; 2. reader isCurrent state also before adding the doc and after re-opening; 3. searched terms vs. added terms - might not be related to concurrency at all. Finally perhaps post here the code so people can take a look. Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 12:45:08: I'm trying what should be the dumbest possible example of concurrency management with 2.0 in Java with an ordinary FSDirectory. I create an IndexWriter from a pathname, an IndexReader from the same pathname, and an IndexSearcher from the reader. I add one document. I call isCurrent() on the reader. It says, 'false'. So, I close the reader and the searcher, and I create a new reader and a new searcher. I search for the document, and I don't find it. I must be missing something simple. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: isCurrent says no, but contents still invisible
And don't forget that you need to close and re-open the reader to pick up the changes... [EMAIL PROTECTED] G. On 1/10/07, Benson Margulies [EMAIL PROTECTED] wrote: #2 is a possible issue. I stared at the code some more: The test case adds up to : Create all the objects. Add three docs. Add a fourth doc. Do a query aimed at the fourth doc. isCurrent() returns false. Close reader/searcher, open reader/searcher, numDocs() in the reader returns 3. Not 4. However, reading your message carefully, I realize that I'm probably fundamentally misguided. There's no flush() API on a writer, so, of course, the only possible way for a reader to see current contents is if the writer gets closed and reopened. I keep trying to cook up some scheme in which this is not true, but, with the stock classes, it now seems self-evident to me that it has to be true. I'll put in the requisite code, and slink away. -Original Message- From: Doron Cohen [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 4:07 PM To: java-user@lucene.apache.org Subject: Re: isCurrent says no, but contents still invisible That's strange. Since you don't close the writer usually adding the doc would not modify the index (unless adding the doc triggered a merge). You may want to check that: 1. writer and reader really opened against the same path; 2. reader isCurrent state also before adding the doc and after re-opening; 3. searched terms vs. added terms - might not be related to concurrency at all. Finally perhaps post here the code so people can take a look. Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 12:45:08: I'm trying what should be the dumbest possible example of concurrency management with 2.0 in Java with an ordinary FSDirectory. I create an IndexWriter from a pathname, an IndexReader from the same pathname, and an IndexSearcher from the reader. I add one document. I call isCurrent() on the reader. It says, 'false'. So, I close the reader and the searcher, and I create a new reader and a new searcher. I search for the document, and I don't find it. I must be missing something simple. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: isCurrent says no, but contents still invisible
Yea, that part I got. -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 5:22 PM To: java-user@lucene.apache.org Subject: Re: isCurrent says no, but contents still invisible And don't forget that you need to close and re-open the reader to pick up the changes... [EMAIL PROTECTED] G. On 1/10/07, Benson Margulies [EMAIL PROTECTED] wrote: #2 is a possible issue. I stared at the code some more: The test case adds up to : Create all the objects. Add three docs. Add a fourth doc. Do a query aimed at the fourth doc. isCurrent() returns false. Close reader/searcher, open reader/searcher, numDocs() in the reader returns 3. Not 4. However, reading your message carefully, I realize that I'm probably fundamentally misguided. There's no flush() API on a writer, so, of course, the only possible way for a reader to see current contents is if the writer gets closed and reopened. I keep trying to cook up some scheme in which this is not true, but, with the stock classes, it now seems self-evident to me that it has to be true. I'll put in the requisite code, and slink away. -Original Message- From: Doron Cohen [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 4:07 PM To: java-user@lucene.apache.org Subject: Re: isCurrent says no, but contents still invisible That's strange. Since you don't close the writer usually adding the doc would not modify the index (unless adding the doc triggered a merge). You may want to check that: 1. writer and reader really opened against the same path; 2. reader isCurrent state also before adding the doc and after re-opening; 3. searched terms vs. added terms - might not be related to concurrency at all. Finally perhaps post here the code so people can take a look. Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 12:45:08: I'm trying what should be the dumbest possible example of concurrency management with 2.0 in Java with an ordinary FSDirectory. I create an IndexWriter from a pathname, an IndexReader from the same pathname, and an IndexSearcher from the reader. I add one document. I call isCurrent() on the reader. It says, 'false'. So, I close the reader and the searcher, and I create a new reader and a new searcher. I search for the document, and I don't find it. I must be missing something simple. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: isCurrent says no, but contents still invisible
Oh, boy, what a mistake. I thought I was being clever by creating a Directory object. All that did was prevent the writer from ever quite flushing because I wasn't closing THAT. -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 5:22 PM To: java-user@lucene.apache.org Subject: Re: isCurrent says no, but contents still invisible And don't forget that you need to close and re-open the reader to pick up the changes... [EMAIL PROTECTED] G. On 1/10/07, Benson Margulies [EMAIL PROTECTED] wrote: #2 is a possible issue. I stared at the code some more: The test case adds up to : Create all the objects. Add three docs. Add a fourth doc. Do a query aimed at the fourth doc. isCurrent() returns false. Close reader/searcher, open reader/searcher, numDocs() in the reader returns 3. Not 4. However, reading your message carefully, I realize that I'm probably fundamentally misguided. There's no flush() API on a writer, so, of course, the only possible way for a reader to see current contents is if the writer gets closed and reopened. I keep trying to cook up some scheme in which this is not true, but, with the stock classes, it now seems self-evident to me that it has to be true. I'll put in the requisite code, and slink away. -Original Message- From: Doron Cohen [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 4:07 PM To: java-user@lucene.apache.org Subject: Re: isCurrent says no, but contents still invisible That's strange. Since you don't close the writer usually adding the doc would not modify the index (unless adding the doc triggered a merge). You may want to check that: 1. writer and reader really opened against the same path; 2. reader isCurrent state also before adding the doc and after re-opening; 3. searched terms vs. added terms - might not be related to concurrency at all. Finally perhaps post here the code so people can take a look. Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 12:45:08: I'm trying what should be the dumbest possible example of concurrency management with 2.0 in Java with an ordinary FSDirectory. I create an IndexWriter from a pathname, an IndexReader from the same pathname, and an IndexSearcher from the reader. I add one document. I call isCurrent() on the reader. It says, 'false'. So, I close the reader and the searcher, and I create a new reader and a new searcher. I search for the document, and I don't find it. I must be missing something simple. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: isCurrent says no, but contents still invisible
Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 14:26:42: Oh, boy, what a mistake. I thought I was being clever by creating a Directory object. All that did was prevent the writer from ever quite flushing because I wasn't closing THAT. No need to close the directory object for that. Closing the IndexWriter will do. I think you will find Lucene In Action book very useful in understanding the life-cycle of an index. Or you may want to start with the javadocs intro at http://lucene.apache.org/java/docs/api (see the bottom of that page) and explore the API from there. In particular: http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWriter.html#addDocument(org.apache.lucene.document.Document) http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWriter.html#close() Also relevant are FAQ entries: http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-6c56b0449d114826586940dcc6fe51582676a36e (concurrency) http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-3558e5121806fb4fce80fc022d889484a9248b71 (why-no-hits) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: isCurrent says no, but contents still invisible
If all is well with the lifecycle, should IndexReader.numDocs() return an accurate count? -Original Message- From: Doron Cohen [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 5:54 PM To: java-user@lucene.apache.org Subject: RE: isCurrent says no, but contents still invisible Benson Margulies [EMAIL PROTECTED] wrote on 10/01/2007 14:26:42: Oh, boy, what a mistake. I thought I was being clever by creating a Directory object. All that did was prevent the writer from ever quite flushing because I wasn't closing THAT. No need to close the directory object for that. Closing the IndexWriter will do. I think you will find Lucene In Action book very useful in understanding the life-cycle of an index. Or you may want to start with the javadocs intro at http://lucene.apache.org/java/docs/api (see the bottom of that page) and explore the API from there. In particular: http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWrit er.html#addDocument(org.apache.lucene.document.Document) http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWrit er.html#close() Also relevant are FAQ entries: http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-6c56b0449d114826586 940dcc6fe51582676a36e (concurrency) http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-3558e5121806fb4fce8 0fc022d889484a9248b71 (why-no-hits) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: isCurrent says no, but contents still invisible
OK, all is well. I had a truly embarrassing logic error that I introduced while getting all the closing and opening straightened out. Thanks for the patience. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
