Just ran some tests .. it appears that the problem is in the sorting ..
i.e.
//hits = searcher.search(query, new Sort("sid", true)); -> 17 secs
//hits = searcher.search(query, new Sort("sid", false)); -> 17 secs
hits = searcher.search(query); -> less than 1 sec ..
am trying something out .. .. will keep you posted
On 8/20/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:
This is why a warming strategy like Solr takes is very valuable. The
searchable index is always serving up requests as fast as Lucene
works, which is achieved by warming a new IndexSearcher with searches/
sorts/filter creating/etc before it is swapped into use.
Erik
On Aug 20, 2006, at 5:35 AM, M A wrote:
> Ok I get your point, this still however means the first search on
> the new
> searcher will take a huge amount of time .. given that this is
> happening now
> ..
>
> i.e. new search -> new query -> get hits ->20+ secs .. this
> happens every 5
> mins or so ..
>
> although subsequent searches may be quicker ..
>
> Am i to assume for a first search the amount of time is ok -> ..
> seems like
> a long time to me ..?
>
> The other thing is the sorting is fixed .. it never changes .. it
> is always
> sorted by the same field ..
>
> i just built the entire index and it still takes ages .,..
>
>
>
>
>
>
>
>
> On 8/20/06, Chris Hostetter <[EMAIL PROTECTED]> wrote:
>>
>>
>> : This is because the index is updated every 5 mins or so, due to the
>> incoming
>> : feed of stories ..
>> :
>> : When you say iteration, i take it you mean, search request, well
>> for
>> each
>> : search that is conducted I create a new one .. search reader
>> that is ..
>>
>> yeah ... i ment iteration of your test. don't do that.
>>
>> if the index is updated every 5 minutes, then open a new searcher
>> every 5
>> minutes -- and reuse it for theentire 5 minutes. if it's updated
>> "sparadically throughout the day" then open a search, and keep
>> using it
>> untill the index is udated, then open a new one.
>>
>> reusing an indexsearcher as long as possible is one of biggest
>> factors of
>> Lucene applications.
>>
>> :
>> :
>> :
>> : On 8/19/06, Chris Hostetter <[EMAIL PROTECTED]> wrote:
>> : >
>> : >
>> : > : hits = searcher.search(query, new Sort("sid", true));
>> : >
>> : > you don't show where searcher is initialized, and you don't
>> clarify
>> how
>> : > you are timing your multiple iterations -- i'm going to guess
>> that you
>> are
>> : > opening a new searcher every iteration right?
>> : >
>> : > sorting on a field requires pre-computing an array of
>> information for
>> that
>> : > field -- this is both time and space expensive, and is cached per
>> : > IndexReader/IndexSearcher -- so if you reuse the same searcher
>> and
>> time
>> : > multiple iterations you'll find that hte first iteration might be
>> somewhat
>> : > slow, but the rest should be very fast.
>> : >
>> : >
>> : >
>> : > -Hoss
>> : >
>> : >
>> : >
>> ---------------------------------------------------------------------
>> : > To unsubscribe, e-mail: [EMAIL PROTECTED]
>> : > For additional commands, e-mail: [EMAIL PROTECTED]
>> : >
>> : >
>> :
>>
>>
>>
>> -Hoss
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
