Re: Slow import from MsSQL and down cluster during process

2018-10-24 Thread Deepak Goel
Please check if there is a deadlock happening by taking heap dumps Deepak "The greatness of a nation can be judged by the way its animals are treated. Please consider stopping the cruelty by becoming a Vegan" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Daniel Carrasco
Thanks for all, I'll try later ;) Greetings!!. El mié., 24 oct. 2018 a las 7:13, Walter Underwood () escribió: > We handle request rates at a few thousand requests/minute with an 8 GB > heap. 95th percentile response time is 200 ms. Median (cached) is 4 ms. > > An oversized heap will hurt your

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Walter Underwood
We handle request rates at a few thousand requests/minute with an 8 GB heap. 95th percentile response time is 200 ms. Median (cached) is 4 ms. An oversized heap will hurt your query performance because everything stops for the huge GC. RAM is still a thousand times faster than SSD, so you want

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Daniel Carrasco
Hello, I've set that heap size because the solr receives a lot of queries every second and I want to cache as much as possible. Also I'm not sure about the number of documents in the collection, but the webpage have a lot of products. About store the index data in RAM is just an expression. The

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Shawn Heisey
On 10/23/2018 7:15 AM, Daniel Carrasco wrote: Hello, Thanks for your response. We've already thought about that and doubled the instances. Just now for every Solr instance we've 60GB of RAM (40GB configured on Solr), and a 16 Cores CPU. The entire Data can be stored on RAM and will not fill

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Daniel Carrasco
Hello, Thanks for your response. We've already thought about that and doubled the instances. Just now for every Solr instance we've 60GB of RAM (40GB configured on Solr), and a 16 Cores CPU. The entire Data can be stored on RAM and will not fill the RAM (of course talking about raw data, not

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Chris Ulicny
Dan, Do you have any idea on the resource usage for the hosts when Solr starts to become unresponsive? It could be that you need more resources or better AWS instances for the hosts. We had what sounds like a similar scenario when attempting to move one of our solrcloud instances to a cloud

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Daniel Carrasco
Hi, El mar., 23 oct. 2018 a las 10:18, Charlie Hull () escribió: > On 23/10/2018 02:57, Daniel Carrasco wrote: > > annoyingHello, > > > > I've a Solr Cluster that is created with 7 machines on AWS instances. The > > Solr version is 7.2.1 (b2b6438b37073bee1fca40374e85bf91aa457c0b) and all > >

Re: Slow import from MsSQL and down cluster during process

2018-10-23 Thread Charlie Hull
On 23/10/2018 02:57, Daniel Carrasco wrote: annoyingHello, I've a Solr Cluster that is created with 7 machines on AWS instances. The Solr version is 7.2.1 (b2b6438b37073bee1fca40374e85bf91aa457c0b) and all nodes are running on NTR mode and I've a replica by node (7 replicas). One node is used

Slow import from MsSQL and down cluster during process

2018-10-22 Thread Daniel Carrasco
annoyingHello, I've a Solr Cluster that is created with 7 machines on AWS instances. The Solr version is 7.2.1 (b2b6438b37073bee1fca40374e85bf91aa457c0b) and all nodes are running on NTR mode and I've a replica by node (7 replicas). One node is used to import, and the rest are just for serve