Re: Re[4]: solr performance
Hi, I was reading this post and I wondering how can I parallelize document processing??? Thanks Erik Erik Hatcher wrote: On Feb 21, 2007, at 4:25 PM, Jack L wrote: couple of times today at around 158 documents / sec. This is not bad at all. How about search performance? How many concurrent queries have people been having? What does the response time look like? I'm the only user :) What I've done is a proof-of-concept for our library. We have 3.7M records that I've indexed and faceted. Search performance (in my unrealistic single user scenario) is blazing (50ms or so) for purely full-text queries. For queries that return facets, the response times are actually quite good too (~900ms, or less depending on the request) - provided the filter cache is warmed and large enough. This is running on my laptop (MacBook Pro, 2GB RAM, 1.83GHz) - I'm sure on a beefier box it'll only get better. Thanks to the others that clarified. I run my indexers in parallel... but a single instance of Solr (which in turn handles requests in parallel as well). Do you feel if multi-threaded posting is helpful? It depends. If the data processing can be parallelized and your hardware supports it, it can certainly make a big difference... it did in my case. Both CPUs were cooking during my parallel indexing runs. Erik -- View this message in context: http://www.nabble.com/solr-performance-tp9055437p20833421.html Sent from the Solr - User mailing list archive at Nabble.com.
Re[4]: solr performance
Thanks for all who replied. my number 1000 was per minute, not second! I can't read! :-p couple of times today at around 158 documents / sec. This is not bad at all. How about search performance? How many concurrent queries have people been having? What does the response time look like? Thanks to the others that clarified. I run my indexers in parallel... but a single instance of Solr (which in turn handles requests in parallel as well). Do you feel if multi-threaded posting is helpful? I suppose when solr does indexing, it's bound more on solr indexer than the poster? Jack __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Re[4]: solr performance
On 2/21/07, Jack L [EMAIL PROTECTED] wrote: Thanks to the others that clarified. I run my indexers in parallel... but a single instance of Solr (which in turn handles requests in parallel as well). Do you feel if multi-threaded posting is helpful? I suppose when solr does indexing, it's bound more on solr indexer than the poster? It certainly is bound more on solr than the poster, but I've found multithreading beneficial as it removes whatever latency factors might exist--http connections, xml parsing, i/o, the poster, etc. For us, concurrent analysis was less of a gain, but then again our analysis is relatively light. -Mike
Re: Re[4]: solr performance
On Feb 21, 2007, at 4:25 PM, Jack L wrote: couple of times today at around 158 documents / sec. This is not bad at all. How about search performance? How many concurrent queries have people been having? What does the response time look like? I'm the only user :) What I've done is a proof-of-concept for our library. We have 3.7M records that I've indexed and faceted. Search performance (in my unrealistic single user scenario) is blazing (50ms or so) for purely full-text queries. For queries that return facets, the response times are actually quite good too (~900ms, or less depending on the request) - provided the filter cache is warmed and large enough. This is running on my laptop (MacBook Pro, 2GB RAM, 1.83GHz) - I'm sure on a beefier box it'll only get better. Thanks to the others that clarified. I run my indexers in parallel... but a single instance of Solr (which in turn handles requests in parallel as well). Do you feel if multi-threaded posting is helpful? It depends. If the data processing can be parallelized and your hardware supports it, it can certainly make a big difference... it did in my case. Both CPUs were cooking during my parallel indexing runs. Erik
