Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Here you go. No :of cpus per node - 16 on most hosts. No :of disks per host - 1 dedicated disk per SOLR instance as we are pretty write heavy. Vms per host- on most hosts 6 instances running at 4GB RAM each Gc params - -XX:+HeapDumpOnOutOfMemoryError-XX:SurvivorRatio=6-XX:PermSize=512m-XX:NewSize=1024m-XX:MaxPermSize=512m-XX:MaxNewSize=1024m-XX:MaxGCPauseMillis=20-XX:+PrintGCDateStamps-XX:+PrintGCDetails-XX:-TraceClassUnloading-verbose:gc-XX:GCTimeRatio=99-XX:+UseConcMarkSweepGC-Xss512k-Xmx4096m-Xms4096m No :of cores per instance 100k Sent from Alto - altomail.com From: noble.p...@gmail.comnoble.p...@gmail.com To: solr-user@lucene.apache.org Sent: Thursday, May 23, 2013 Subject: Re: Upgrading from SOLR 3.5 to 4.2.1 Results. Actually , It's pretty high end for most of the users. Rishi, u can post the real h/w details and our typical deployment . No :of cpus per node No :of disks per host Vms per host Gc params No :of cores per instance Noble Paul Sent from phone On 21 May 2013 01:47, Rishi Easwaran rishi.easwa...@aol.com wrote: No, we just upgraded to 4.2.1. With the size of our complex and effort required apply our patches and rollout, our upgrades are not that often. -Original Message- From: Noureddine Bouhlel nouredd...@ecotour.com To: solr-user solr-user@lucene.apache.org Sent: Mon, May 20, 2013 3:36 pm Subject: Re: Upgrading from SOLR 3.5 to 4.2.1 Results. Hi Rishi, Have you done any tests with Solr 4.3 ? Regards, Cordialement, BOUHLEL Noureddine On 17 May 2013 21:29, Rishi Easwaran rishi.easwa...@aol.com wrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Actually , It's pretty high end for most of the users. Rishi, u can post the real h/w details and our typical deployment . No :of cpus per node No :of disks per host Vms per host Gc params No :of cores per instance Noble Paul Sent from phone On 21 May 2013 01:47, Rishi Easwaran rishi.easwa...@aol.com wrote: No, we just upgraded to 4.2.1. With the size of our complex and effort required apply our patches and rollout, our upgrades are not that often. -Original Message- From: Noureddine Bouhlel nouredd...@ecotour.com To: solr-user solr-user@lucene.apache.org Sent: Mon, May 20, 2013 3:36 pm Subject: Re: Upgrading from SOLR 3.5 to 4.2.1 Results. Hi Rishi, Have you done any tests with Solr 4.3 ? Regards, Cordialement, BOUHLEL Noureddine On 17 May 2013 21:29, Rishi Easwaran rishi.easwa...@aol.com wrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Rishi: Thanks very much for taking the time to post this, we're always looking for before/after numbers! Erick On Sat, May 18, 2013 at 11:34 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: Awesome news Rishi! Looking forward to your SolrCloud updates. On Sat, May 18, 2013 at 12:59 AM, Rishi Easwaran rishi.easwa...@aol.comwrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi. -- Regards, Shalin Shekhar Mangar.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Hi Rishi, Have you done any tests with Solr 4.3 ? Regards, Cordialement, BOUHLEL Noureddine On 17 May 2013 21:29, Rishi Easwaran rishi.easwa...@aol.com wrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Sure Shalin, hopefully soon. -Original Message- From: Shalin Shekhar Mangar shalinman...@gmail.com To: solr-user solr-user@lucene.apache.org Sent: Sat, May 18, 2013 11:35 pm Subject: Re: Upgrading from SOLR 3.5 to 4.2.1 Results. Awesome news Rishi! Looking forward to your SolrCloud updates. On Sat, May 18, 2013 at 12:59 AM, Rishi Easwaran rishi.easwa...@aol.comwrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi. -- Regards, Shalin Shekhar Mangar.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
We use commodity H/W which we procured over the years as our complex grew. Running on jdk6 with tomcat 5. (Planning to upgrade to jdk7 and tomcat7 soon). We run them with about 4GB heap. Using CMS GC. -Original Message- From: adityab aditya_ba...@yahoo.com To: solr-user solr-user@lucene.apache.org Sent: Sat, May 18, 2013 10:37 am Subject: Re: Upgrading from SOLR 3.5 to 4.2.1 Results. These numbers are really great. Would you mind sharing your h/w configuration and JVM params thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Upgrading-from-SOLR-3-5-to-4-2-1-Results-tp4064266p4064370.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
No, we just upgraded to 4.2.1. With the size of our complex and effort required apply our patches and rollout, our upgrades are not that often. -Original Message- From: Noureddine Bouhlel nouredd...@ecotour.com To: solr-user solr-user@lucene.apache.org Sent: Mon, May 20, 2013 3:36 pm Subject: Re: Upgrading from SOLR 3.5 to 4.2.1 Results. Hi Rishi, Have you done any tests with Solr 4.3 ? Regards, Cordialement, BOUHLEL Noureddine On 17 May 2013 21:29, Rishi Easwaran rishi.easwa...@aol.com wrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
These numbers are really great. Would you mind sharing your h/w configuration and JVM params thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Upgrading-from-SOLR-3-5-to-4-2-1-Results-tp4064266p4064370.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Rishi, Fantastic! Thank you so very much for sharing the details. Jason On May 17, 2013, at 12:29 PM, Rishi Easwaran rishi.easwa...@aol.com wrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
Awesome news Rishi! Looking forward to your SolrCloud updates. On Sat, May 18, 2013 at 12:59 AM, Rishi Easwaran rishi.easwa...@aol.comwrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi. -- Regards, Shalin Shekhar Mangar.
Upgrading from SOLR 3.5 to 4.2.1 Results.
Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
Re: Upgrading from SOLR 3.5 to 4.2.1 Results.
This is great; data like this is rare. Can you tell us any hardware or throughput numbers? On 05/17/2013 12:29 PM, Rishi Easwaran wrote: Hi All, Its Friday 3:00pm, warm sunny outside and it was a good week. Figured I'd share some good news. I work for AOL mail team and we use SOLR for our mail search backend. We have been using it since pre-SOLR 1.4 and strong supporters of SOLR community. We deal with millions indexes and billions of requests a day across our complex. We finished full rollout of SOLR 4.2.1 into our production last week. Some key highlights: - ~75% Reduction in Search response times - ~50% Reduction in SOLR Disk busy , which in turn helped with ~90% Reduction in errors - Garbage collection total stop reduction by over 50% moving application throughput into the 99.8% - 99.9% range - ~15% reduction in CPU usage We did not tune our application moving from 3.5 to 4.2.1 nor update java. For the most part it was a binary upgrade, with patches for our special use case. Now going forward we are looking at prototyping SOLR Cloud for our search system, upgrade java and tomcat, tune our application further. Lots of fun stuff :) Have a great weekend everyone. Thanks, Rishi.
