Re: What's the bottleneck?

2008-09-17 Thread Sean Timm
The HitCollector used by the Searcher is wrapped by a TimeLimitedCollector http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc//org/apache/lucene/search/TimeLimitedCollector.html which times out search requests that take longer than the maximum allowed search time limit during the

RE: What's the bottleneck?

2008-09-15 Thread r.prieto
Hi Jason, I'd like to know how you solved the problem. could you post the solution?? Thanks Raúl -Mensaje original- De: Jason Rennie [mailto:[EMAIL PROTECTED] Enviado el: jueves, 11 de septiembre de 2008 21:58 Para: solr-user@lucene.apache.org Asunto: Re: What's the bottleneck? On Thu

Re: What's the bottleneck?

2008-09-12 Thread Jeryl Cook
I think you should justs break up your index across boxes and do a federated search across them... since you mentioned you have a single machine.. Jeryl Cook /^\ Pharaoh /^\ http://pharaohofkush.blogspot.com/ Whether we bring our enemies to justice, or bring justice to our enemies, justice will

RE: What's the bottleneck?

2008-09-12 Thread r.prieto
Para: solr-user@lucene.apache.org Asunto: Re: What's the bottleneck? On Thu, Sep 11, 2008 at 1:29 PM, [EMAIL PROTECTED] wrote: what is your index configuration??? Not sure what you mean. We're using 1.2, though we've tested with a recent nightly and didn't see a significant change

Re: What's the bottleneck?

2008-09-12 Thread Grant Ingersoll
The bottleneck may simply be there are a lot of docs to score since you are using fairly common terms. Also, what file format (compound, non-compound) are you using? Is it optimized? Have you profiled your app for these queries? When you say the query is longer, define longer... 5

Re: What's the bottleneck?

2008-09-12 Thread Jason Rennie
Thanks for all the replies! Mike: we're not using pf. Our qf is always status:0. The status field is 0 for all good docs (90%+) and some other integer for any docs we don't want returned. Jeyrl: federated search is definitely something we'll consider. On Fri, Sep 12, 2008 at 8:39 AM, Grant

Re: What's the bottleneck?

2008-09-12 Thread Ken Krugler
Thanks for all the replies! Mike: we're not using pf. Our qf is always status:0. The status field is 0 for all good docs (90%+) and some other integer for any docs we don't want returned. Jeyrl: federated search is definitely something we'll consider. On Fri, Sep 12, 2008 at 8:39 AM, Grant

Re: What's the bottleneck?

2008-09-12 Thread Otis Gospodnetic
] To: solr-user@lucene.apache.org Sent: Friday, September 12, 2008 2:17:28 PM Subject: Re: What's the bottleneck? Thanks for all the replies! Mike: we're not using pf. Our qf is always status:0. The status field is 0 for all good docs (90%+) and some other integer for any docs we don't want

Re: What's the bottleneck?

2008-09-12 Thread Grant Ingersoll
See also https://issues.apache.org/jira/browse/SOLR-502 (timeout searches) and https://issues.apache.org/jira/browse/LUCENE-997 This is committed on trunk and will be in 1.3. Don't ask me how it works, b/c I haven't tried it yet, but maybe Sean Timm or someone can help out. I'm not sure

Re: What's the bottleneck?

2008-09-11 Thread Mark Miller
Jason Rennie wrote: We have a 14 million document index that we only use for querying (optimized, read-only). When we issue queries that have few, relatively rare words, the query returns quickly. However, when the query is longer and uses more common words (hitting, say, ~1 million docs), it

Re: What's the bottleneck?

2008-09-11 Thread Jason Rennie
On Thu, Sep 11, 2008 at 11:54 AM, Mark Miller [EMAIL PROTECTED] wrote: What kind of traffic are you getting when it takes seconds? 1 request? 12? I'd estimate concurrency around 3, though the speed doesn't change much when we run the same query on a server with zero traffic. Jason

RE: What's the bottleneck?

2008-09-11 Thread r.prieto
Hi Jason, some questions .. what is your index configuration??? What is your average size form the returned fields ??? How much memory have your System ?? Do you have long fieds who is returned in the queries ? Do you have actívate the Highlighting in the request ? Are you using multi-value filed

Re: What's the bottleneck?

2008-09-11 Thread Jason Rennie
On Thu, Sep 11, 2008 at 1:29 PM, [EMAIL PROTECTED] wrote: what is your index configuration??? Not sure what you mean. We're using 1.2, though we've tested with a recent nightly and didn't see a significant change in performance... What is your average size form the returned fields ???

Re: What's the bottleneck?

2008-09-11 Thread Mike Klaas
On 11-Sep-08, at 8:24 AM, Jason Rennie wrote: We have a 14 million document index that we only use for querying (optimized, read-only). When we issue queries that have few, relatively rare words, the query returns quickly. However, when the query is longer and uses more common words