Re: Performance of Solr on different Platforms
Eswar, This link would give you a fair idea of how Solr is used by some of the sites/companies - http://wiki.apache.org/solr/SolrPerformanceData Rishabh On Nov 20, 2007 10:49 AM, Eswar K [EMAIL PROTECTED] wrote: In our case, the load is kind of distributed. On an average, the QPS could be much less than that. 1000 qps could be the peak load ever expected could ever reach. However the number of documents going to be in the range of 2 - 20 million documents. We would possibly distribute the indexes to different solr instances and possibly direct it accordingly to reduce the QPS. - Eswar On Nov 20, 2007 10:42 AM, Walter Underwood [EMAIL PROTECTED] wrote: 1000 qps is a lot of load, at least 30M queries/day. We are running dual CPU Power P5 machines and getting about 80 qps with worst case response times of 5 seconds. 90% of responses are under 70 msec. Our expected peak load is 300 qps on our back-end Solr farm. We execute multiple back-end queries for each query page. With N+1 sizing (full throughput with one server down), we have five servers to do that. We have a separate server for indexing and use the Solr distribution scripts. We have a relatively small index, about 250K docs. wunder On 11/19/07 8:48 PM, Eswar K [EMAIL PROTECTED] wrote: Its not going to hit 1000 all the time, its the expected peak value. I guess for distributing the load we should be using collections and I was looking at the collections documentation ( http://wiki.apache.org/solr/CollectionDistribution) . - Eswar On Nov 20, 2007 12:07 AM, Matthew Runo [EMAIL PROTECTED] wrote: I'd think that any platform that can run Java would be fine to run SOLR on. Maybe this is more a question of preferred platforms for Java deployments? That is quite the load for SOLR though, you may find that you want more than one server. Do you mean that you're expecting about 1000 QPS over an index with up to 20 million documents? --Matthew On Nov 19, 2007, at 6:00 AM, Eswar K wrote: All, Can you give some information on this or atleast let me know where I can find this information if its already listed out anywhere. Regards, Eswar On Nov 18, 2007 9:45 PM, Eswar K [EMAIL PROTECTED] wrote: Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
Re: Performance of Solr on different Platforms
Most of Sematext's customers seem to be RH fans. I've seen some Ubuntu, some Debian, and some SuSe users. RH feels safe. :) Some use Solaris. Some are going crazy with Xen, putting everything in VMs. RAM - as much as you can afford, as usual. CPU - AMD Opterons performed the best last time I benchmarked a bunch of different type of hardware - stay away from Sun Niagara servers for Lucene/Solr. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Eswar K [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Sunday, November 18, 2007 11:15:48 AM Subject: Performance of Solr on different Platforms Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
Re: Performance of Solr on different Platforms
All, Can you give some information on this or atleast let me know where I can find this information if its already listed out anywhere. Regards, Eswar On Nov 18, 2007 9:45 PM, Eswar K [EMAIL PROTECTED] wrote: Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
Re: Performance of Solr on different Platforms
Its not going to hit 1000 all the time, its the expected peak value. I guess for distributing the load we should be using collections and I was looking at the collections documentation ( http://wiki.apache.org/solr/CollectionDistribution) . - Eswar On Nov 20, 2007 12:07 AM, Matthew Runo [EMAIL PROTECTED] wrote: I'd think that any platform that can run Java would be fine to run SOLR on. Maybe this is more a question of preferred platforms for Java deployments? That is quite the load for SOLR though, you may find that you want more than one server. Do you mean that you're expecting about 1000 QPS over an index with up to 20 million documents? --Matthew On Nov 19, 2007, at 6:00 AM, Eswar K wrote: All, Can you give some information on this or atleast let me know where I can find this information if its already listed out anywhere. Regards, Eswar On Nov 18, 2007 9:45 PM, Eswar K [EMAIL PROTECTED] wrote: Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
Re: Performance of Solr on different Platforms
In our case, the load is kind of distributed. On an average, the QPS could be much less than that. 1000 qps could be the peak load ever expected could ever reach. However the number of documents going to be in the range of 2 - 20 million documents. We would possibly distribute the indexes to different solr instances and possibly direct it accordingly to reduce the QPS. - Eswar On Nov 20, 2007 10:42 AM, Walter Underwood [EMAIL PROTECTED] wrote: 1000 qps is a lot of load, at least 30M queries/day. We are running dual CPU Power P5 machines and getting about 80 qps with worst case response times of 5 seconds. 90% of responses are under 70 msec. Our expected peak load is 300 qps on our back-end Solr farm. We execute multiple back-end queries for each query page. With N+1 sizing (full throughput with one server down), we have five servers to do that. We have a separate server for indexing and use the Solr distribution scripts. We have a relatively small index, about 250K docs. wunder On 11/19/07 8:48 PM, Eswar K [EMAIL PROTECTED] wrote: Its not going to hit 1000 all the time, its the expected peak value. I guess for distributing the load we should be using collections and I was looking at the collections documentation ( http://wiki.apache.org/solr/CollectionDistribution) . - Eswar On Nov 20, 2007 12:07 AM, Matthew Runo [EMAIL PROTECTED] wrote: I'd think that any platform that can run Java would be fine to run SOLR on. Maybe this is more a question of preferred platforms for Java deployments? That is quite the load for SOLR though, you may find that you want more than one server. Do you mean that you're expecting about 1000 QPS over an index with up to 20 million documents? --Matthew On Nov 19, 2007, at 6:00 AM, Eswar K wrote: All, Can you give some information on this or atleast let me know where I can find this information if its already listed out anywhere. Regards, Eswar On Nov 18, 2007 9:45 PM, Eswar K [EMAIL PROTECTED] wrote: Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
Re: Performance of Solr on different Platforms
1000 qps is a lot of load, at least 30M queries/day. We are running dual CPU Power P5 machines and getting about 80 qps with worst case response times of 5 seconds. 90% of responses are under 70 msec. Our expected peak load is 300 qps on our back-end Solr farm. We execute multiple back-end queries for each query page. With N+1 sizing (full throughput with one server down), we have five servers to do that. We have a separate server for indexing and use the Solr distribution scripts. We have a relatively small index, about 250K docs. wunder On 11/19/07 8:48 PM, Eswar K [EMAIL PROTECTED] wrote: Its not going to hit 1000 all the time, its the expected peak value. I guess for distributing the load we should be using collections and I was looking at the collections documentation ( http://wiki.apache.org/solr/CollectionDistribution) . - Eswar On Nov 20, 2007 12:07 AM, Matthew Runo [EMAIL PROTECTED] wrote: I'd think that any platform that can run Java would be fine to run SOLR on. Maybe this is more a question of preferred platforms for Java deployments? That is quite the load for SOLR though, you may find that you want more than one server. Do you mean that you're expecting about 1000 QPS over an index with up to 20 million documents? --Matthew On Nov 19, 2007, at 6:00 AM, Eswar K wrote: All, Can you give some information on this or atleast let me know where I can find this information if its already listed out anywhere. Regards, Eswar On Nov 18, 2007 9:45 PM, Eswar K [EMAIL PROTECTED] wrote: Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
Performance of Solr on different Platforms
Hi, I understand that Solr can be used on different Linux flavors. Is there any preferred flavor (Like Red Hat, Ubuntu, etc)? Also what is the kind of configuration of hardware (Processors, RAM, etc) be best suited for the install? We expect to load it with millions of documents (varying from 2 - 20 million). There might be around 1000 concurrent users. Your help in this regard will be appreciated. Regards, Eswar
