Re: Replication slows down massively during high load

2011-03-17 Thread Vadim Kisselmann
Hi Bill, > You could always rsync the index dir and reload (old scripts). I used them previously but was getting problems with them. The application querying the Solr doesn't cause enough load on it to trigger the issue. Yet. > But this is still something we should investigate. Indeed :-) > Se

Re: Replication slows down massively during high load

2011-03-17 Thread Bill Bell
You could always rsync the index dir and reload (old scripts). But this is still something we should investigate. I had this same issue on high load and never really found a solution. Did you try another Nic card? See if the Nic is configured right? Routing? Speed of transfer? Bill Bell Sent fr

Re: Replication slows down massively during high load

2011-03-17 Thread Vadim Kisselmann
On Mar 17, 2011, at 3:19 PM, Shawn Heisey wrote: On 3/17/2011 3:43 AM, Vadim Kisselmann wrote: Unfortunately, this doesn't seem to be the problem. The queries themselves are running fine. The problem is that the replications is crawling when there are many queries going on and that the replication

Re: Replication slows down massively during high load

2011-03-17 Thread Shawn Heisey
On 3/17/2011 3:43 AM, Vadim Kisselmann wrote: Unfortunately, this doesn't seem to be the problem. The queries themselves are running fine. The problem is that the replications is crawling when there are many queries going on and that the replication speed stays low even after the load is gone.

Re: Replication slows down massively during high load

2011-03-17 Thread Vadim Kisselmann
Hello Shawn, Primary assumption:  You have a 64-bit OS and a 64-bit JVM. >Jepp, it's running 64-bit Linux with 64-bit JVM It sounds to me like you're I/O bound, because your machine cannot keep enough of your index in RAM.  Relative to your 100GB index, you only have a maximum of 14G

Re: Replication slows down massively during high load

2011-03-16 Thread Shawn Heisey
On 3/16/2011 6:09 PM, Shawn Heisey wrote: du -hc *x I was looking over the files in an index and I think it needs to include more of the files for a true picture of RAM needs. I get 5.9GB running the following command against a 16GB index. It excludes *.fdt (stored field data) and *.tvf (t

Re: Replication slows down massively during high load

2011-03-16 Thread Shawn Heisey
On 3/16/2011 7:56 AM, Vadim Kisselmann wrote: If the load is low, both slaves replicate with around 100MB/s from master. But when I use Solrmeter (100-400 queries/min) for load tests (over the load balancer), the replication slows down to an unacceptable speed, around 100KB/s (at least that's wh

Replication slows down massively during high load

2011-03-16 Thread Vadim Kisselmann
Hi everyone, I have Solr running on one master and two slaves (load balanced) via Solr 1.4.1 native replication. If the load is low, both slaves replicate with around 100MB/s from master. But when I use Solrmeter (100-400 queries/min) for load tests (over the load balancer), the replication slow