Re: Binary indexing / query efficiency

2009-04-14 Thread Khawaja Shams
document id, and you can use this value to extract the string from the lookup array once you are done counting. Perhaps the Lucene experts can shed light on a better approach. You may also want to look at SOLR for faceted searching support :). HTH. Regards, Khawaja Shams On Tue, Apr 14, 2009 at 11:12

Re: Exceptions in merge thread (while optimizing) causing problems with subsequent reopens

2009-04-09 Thread Khawaja Shams
gt; index, up until when the optimize error happens, and post back? > > Mike > > On Thu, Apr 9, 2009 at 8:33 PM, Khawaja Shams wrote: > > Hi Michael, > > Thanks for the quick response. I only have one IndexWriter, and there > are > > no other processes accessing th

Re: Exceptions in merge thread (while optimizing) causing problems with subsequent reopens

2009-04-09 Thread Khawaja Shams
These are serious corruption exceptions. > > Is it at all possible two writers are accessing the index at the same time? > > Can you describe more about how you're using Lucene? > > Mike > > On Thu, Apr 9, 2009 at 7:59 PM, Khawaja Shams wrote: > > Hello, > > I

Exceptions in merge thread (while optimizing) causing problems with subsequent reopens

2009-04-09 Thread Khawaja Shams
Hello, I am having a problem with reopening the IndexReader with Lucene 2.4 ( I updated to 2.4.1, but still no luck). The exception is preceded by an exception in optimizing the index. I am not reopening the reader while the commit or optimization is going on in the writer (optimizing happens in

Re: Sorting documents without a query

2008-12-06 Thread Khawaja Shams
Hi Shivaraj, I would recommend John's approach and let Lucene deal with the sorting. Nonetheless, you can traverse through the terms by using the following method: http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/index/IndexReader.html#terms(org.apache.lucene.index.Term) Basically, yo

Re: lucene nicking my memory ?

2008-12-04 Thread Khawaja Shams
but it is unclear if the queries are returning at that point. Lastly, I highly recommend against ever making the gc requests. Regards, Khawaja Shams On Thu, Dec 4, 2008 at 12:30 AM, Khawaja Shams <[EMAIL PROTECTED]> wrote: > Magnus, If you get a chance, can you try setting a di

Re: lucene nicking my memory ?

2008-12-04 Thread Khawaja Shams
Magnus, If you get a chance, can you try setting a different xms and xmx value. For instance, try xms384M and xmx1024M. The "forced" GC [request] will almost always reduce the memory footprint simply because of the weak references that lucene leverages, but I bet subsequent queries are not as

Re: Indexing Names in Lucene -- Thomas = Tom, etc

2008-12-03 Thread Khawaja Shams
e was a canonical way of doing this before building my own list from common sir names. Thanks for your time. Sincerely, Khawaja Shams On Wed, Dec 3, 2008 at 1:42 AM, Ganesh <[EMAIL PROTECTED]> wrote: > If you want to query for Tom, then you need to index the value Tom. Create >

Indexing Names in Lucene -- Thomas = Tom, etc

2008-12-02 Thread Khawaja Shams
addressed, but I was unable to find anything in the archives or on Google. Thanks in advance for the guidance. Regards, Khawaja Shams

Re: Which is faster/better

2008-11-25 Thread Khawaja Shams
On Tue, Nov 25, 2008 at 8:42 AM, Grant Ingersoll <[EMAIL PROTECTED]>wrote: > > On Nov 25, 2008, at 10:46 AM, Michael McCandless wrote: > > If you already have the docId, would you need to/want to do >>> delete-by-Query or even delete-by-Term? Isn't delete-by-id a lot lighter >>> weight since it

Re: Closing Index Reader

2008-10-15 Thread Khawaja Shams
Hello, Thank you for your ideas. While these look promising, it seems like there are many places within Lucene's codebase that are invoking the incRef and decRef methods. Upon a shallow analysis of the code, it seems like I can call close, and the doClose method does not get called until refCoun

Re: distinct field values

2008-10-14 Thread Khawaja Shams
with each tag added as a constraint to find out if there are any matches or you can manage a filter for each value to deduce if it exists in the resultset even faster. Please share your experience as you find more clues on this problem. Regards, Khawaja Shams On Tue, Oct 14, 2008 at 8:15 PM,

Closing Index Reader

2008-10-14 Thread Khawaja Shams
Hello, I am using the reopen method in the IndexReader class. In the case of the IndexReader being updated, I would like to create a new IndexSearcher and close the old IndexReader. When closing an instance of IndexReader, do I have to wait for currently executing searches (through an IndexSearche

Enumerating all the terms of a particular field

2008-10-12 Thread Khawaja Shams
Hello, How can I get a list of all the terms for a particular field? Is the right approach to extend FilteredTermEnum? Thanks!!

Detecting why a collection of documents matched a query

2008-10-12 Thread Khawaja Shams
Hello, I noticed that indexSearcher.explain() method is not supposed to be run for a large collection of documents, so I am looking for an alternative that just explains why a document matched without all the scoring information. Basically, I would like to know which field of the document was resp