Re: Lucene Scalability Question

[Ali Salehi]

Hi,
 Thanks for your previous mail.
 Now I changed the configuration to use merging factor 50. I also disabled
the compound file parameter.
 I still have the search problem I had before, now search takes around 750
msecs for a small set of documents.

    [java] Total Query Processing time (msec) : 38745
    [java] Total No. of Documents : 7,500,000
    [java] Total No. of Executed queries : 50.0
    [java] Execution time per query : 774.9 msec

 The index is optimized and its size is 830 MB.
 Each document has the following terms :
    VSID(integer), data(float), type(short int) , precision (byte).
  The queries are generate in a loop similar to one below :
loop ...
    RangeFilter rq1 = new
RangeFilter("data",”+5.43243243440000”,”+5.43243243449999”true,true);
    RangeFilter rq2 = new RangeFilter("precision",”+0001”,”+0002”,true,true);
    ChainedFilter cf = new ChainedFilter(new
Filter[]{rq2,rq1},ChainedFilter.AND);
    Query query = qp.parse("type:0");
    Hits hits = searcher.search(query,cf);
end loop

 I would like to know if there exist any solution to improve the search
time ?  (I need to insert more than 500 million of these data pages into
lucene)

Thanks,
AliS



> On Monday 08 January 2007 20:33, Ali Salehi wrote:
>
>> Â 1. The search time for simple queries such as precision:\+0002 is
>> really high (4-10 seconds). I want to know if this search time is normal
>
>> Â 2. The search gives TooManyClauses exception when I'm searching for a
>> Â data item with the queries similar to the one below :
>
> Please see the FAQ at http://wiki.apache.org/jakarta-lucene/LuceneFAQ:
> Why am I getting a TooManyClauses exception?
> How do I speed up searching?
>
> If that doesn't help, please re-post you question on the user list.
>
> Regards
>  Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>


**************************************************************
Ali Salehi, LSIR - Distributed Information Systems Laboratory
EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
http://lsirwww.epfl.ch/
email: [EMAIL PROTECTED]
Tel: +41-21-6936656 Fax: +41-21-6938115


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]