Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Yonik Seeley
On Sun, Feb 11, 2018 at 8:47 AM, ~$alpha` wrote: > I have upgraded Solr4.0 Beta to Solr6.6. The Cache results look Awesome but > overall the CPU load on solr6.6 is double the load on solr4.0 and hence I am > not able to roll solr6.6 to 100% of my traffic. > > *Some Key

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread ~$alpha`
cpu utlization is already on higher end, so VM wont seems a solution -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Deepak Goel
One more idea could be is to have multiple vm's (8 cpu each) on your server and load balance them. That would help Solr6 scale nicely On 12 Feb 2018 23:05, "Deepak Goel" wrote: > If the community cannot help, the only way i can think is either to > profile Solr (java) under a

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread ~$alpha`
top command in linux -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Walter Underwood
25 of 32 CPUs loaded is 78% CPU, but I’ve never seen CPU use reported that way. How are you measuring CPU usage? What tool? wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Feb 12, 2018, at 7:08 AM, ~$alpha` wrote: > > I

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Deepak Goel
If the community cannot help, the only way i can think is either to profile Solr (java) under a load test to find the problem. You could also use an APM. On 12 Feb 2018 23:00, "~$alpha`" wrote: > Yes, but how to move ahead now. > Its strange solr4 is better behaving

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread ~$alpha`
Yes, but how to move ahead now. Its strange solr4 is better behaving than solr6 -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Deepak Goel
This would then mean that solr6 is reaching some kind of saturation (number of threads, etc) at about loads of 60 Hits which then drives the performance of it to be very bad ! Deepak "Please stop cruelty to Animals, help by becoming a Vegan" +91 73500 12833 deic...@gmail.com Facebook:

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Walter Underwood
100% CPU will cause congestion and very slow response. In production, we do not drive Solr over 75% CPU. He is reporting load average, which is a bit harder to interpret. When the load average reaches the number of CPUs, that is probably the beginning of congestion. But I’m less sure about

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread ~$alpha`
I cant test on more as performance is already degraded. Its a 32core system and load 25 means 2500% cpu -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Deepak Goel
Please test for higher number of hits till cpu load reaches 100% On 12 Feb 2018 19:44, "~$alpha`" wrote: > Hits 41 : > Avg response time470ms vs 380ms > CPU Load reaches6 vs 10 > > Hits 82: > Avg response time 500ms vs 620ms

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread ~$alpha`
Hits 41 : Avg response time470ms vs 380ms CPU Load reaches6 vs 10 Hits 82: Avg response time 500ms vs 620ms (solr6 performing bad on peak hours) CPU Load reaches11 vs 25 -- Sent from:

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Deepak Goel
I would suggest to keep the load same both for solr4 and solr6. And then test. Also please post exact concurrent hits On 12 Feb 2018 12:48, "~$alpha`" wrote: When both solr4 and solr6 have concurrent hits: 1. 30 to 40 : Avg response time 470ms vs 380ms Load

Re: Solr4 To Solr6 CPU load issues

2018-02-12 Thread Emir Arnautović
Hi Birender, Do you monitor you heap? Is it possible that you are running close to max heap size and that GC is what is taking CPU? Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 11 Feb 2018, at

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread ~$alpha`
When both solr4 and solr6 have concurrent hits: 1. 30 to 40 : Avg response time 470ms vs 380ms Load 6 vs 10 1. 80 to 90 : Avg response time 500ms vs 620ms (solr6 performing bad on peak hours) Load 11 vs 25 -- Sent from:

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread Deepak Goel
Also can you please post the throughputs for both of your tests On 12 Feb 2018 00:35, "Deepak Goel" wrote: > Yup. Improvement of response time would hurt the cpu usage. The other > thing is more memory usage (cache) which gets included into the cpu usage. > > > > Deepak >

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread Deepak Goel
Yup. Improvement of response time would hurt the cpu usage. The other thing is more memory usage (cache) which gets included into the cpu usage. Deepak "Please stop cruelty to Animals, help by becoming a Vegan" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread ~$alpha`
Other resource is not using the cpu. Its true that response is better to 300ms from 350ms but cpu usage almost doubled? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread Deepak Goel
Two things trouble me: 1. It is a shared resource, so results are unreliable 2. Since cache results have increased, memory access will increase and it will result in an increase in cpu usage. However response times will also improve To support more load you will have to increase server capacity

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread ~$alpha`
Config : 64GB RAM 32 CORE CPU but i have given 20Gb to solr JVM.. Also its a shared resource -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr4 To Solr6 CPU load issues

2018-02-11 Thread Deepak Goel
Can you please give the configuration of your server? On 11 Feb 2018 19:17, "~$alpha`" wrote: > I have upgraded Solr4.0 Beta to Solr6.6. The Cache results look Awesome but > overall the CPU load on solr6.6 is double the load on solr4.0 and hence I > am > not able to roll