Re[2]: Out of memory exception for big indexes

2007-04-25 Thread Artem
Hello Ivan, That was cool news! Thanks! :) The timings are surprisingly good. 10 mln docs sorted in 20s.. cool! Also it looks like sorting algorithm employed by Lucene is quite memory-economic. Not supporting multiple fields is in fact another limitation of my patch. I don't need it so I didn't i

Fwd: Re[2]: Out of memory exception for big indexes

2007-04-09 Thread Artem
Hello Nilesh and all! NB> This seems like a very useful patch. Our application searches over 50 NB> million doc in a 40GB index. We only have simple conjunctive queries NB> on a single field. Currently, the command line search program that NB> prints top-10 results requires at least 200mb memory.

Re: Re[2]: Out of memory exception for big indexes

2007-04-08 Thread Erick Erickson
It *is* a bit confusing, since every search is sorted, kinda Practically, a sorted query is one where you call one of the search methods (on, say, Searcher) with a Sort object, which sorts on one or more of the fields in your index (which ones are used are specified in the (array of) Sort obj

Re: Re[2]: Out of memory exception for big indexes

2007-04-08 Thread Nilesh Bansal
On 4/8/07, Artem <[EMAIL PROTECTED]> wrote: I must note that my patch only helps in lucene-OOM situations related to _sorted_ queries. If this is your case than I think yes it will help. Probably a newbie question, but can you please explain what sorted queries mean? Is simple keyword search a s

Re[2]: Out of memory exception for big indexes

2007-04-08 Thread Artem
Hello Nilesh, Sunday, April 8, 2007, 9:03:06 AM, you wrote: NB> This seems like a very useful patch. Our application searches over 50 NB> million doc in a 40GB index. We only have simple conjunctive queries NB> on a single field. Currently, the command line search program that NB> prints top-10 r