Re: adding documents to an existing index

2010-10-29 Thread Yakob
but anyway,I've done it.thanks for your help. I did look up your source code and find it easier to understand. this is the kind of help that I want. thank you very much then. :-) On 10/27/10, Seth Rosen wrote: > Yakob the boolean in the constructor should be true if you want to create a > NEW in

lucene norms cached twice

2010-10-29 Thread Cabansag, Ronald-Alvin R
We are working with a large readonly lucene index(single segment) with large number of fields and documents and are running into memory usage problems. We found that when using a ReadOnlyDirectoryReader and IndexSearcher created using the same reader, the norms are cached twice - first by the r

Re: lucene norms cached twice

2010-10-29 Thread Michael McCandless
Norms should not normally be loaded twice. Since 2.9, searching is done at the sub-reader level, and so norms should never be loaded for the main reader. But can you describe how you're using Lucene? Mike On Fri, Oct 29, 2010 at 9:27 AM, Cabansag, Ronald-Alvin R wrote: > > We are working with

RE: lucene norms cached twice

2010-10-29 Thread Cabansag, Ronald-Alvin R
We create a single instance of IndexReader using IndexReader.open(new MMapDirectory(file))and a single instance of IndexSearcher using this index reader. Searches in our application are done with two operations. The first operation gets the hit count. We use a QueryWrapperFilter.getDocIdSet(ind

Re: lucene norms cached twice

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 3:32 PM, Cabansag, Ronald-Alvin R wrote: > We use a QueryWrapperFilter.getDocIdSet(indexReader) to get the DocIdSet and > compute the hit count using its iterator. If you want to avoid double-caching of norms, then you should call getDocIdSet() for each segment reader, no

RE: lucene norms cached twice

2010-10-29 Thread Cabansag, Ronald-Alvin R
Yonik, Thanks for the input. We'll try this out. And you're right - I tried to simplify our first operation's description. -Al Cengage Learning -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Friday, October 29, 2010 3:40 PM To: java-