Re: Read time get worse during dynamic snitch reset

2011-05-11 Thread shimi
I finally found some time to get back to this issue. I turned on the DEBUG log on the StorageProxy and it shows that all of these request are read from the other datacenter. Shimi On Tue, Apr 12, 2011 at 2:31 PM, aaron morton aa...@thelastpickle.comwrote: Something feels odd. From Peters

Re: Read time get worse during dynamic snitch reset

2011-04-12 Thread shimi
On Tue, Apr 12, 2011 at 12:26 AM, aaron morton aa...@thelastpickle.comwrote: The reset interval clears the latency tracked for each node so a bad node will be read from again. The scores for each node are then updated every 100ms (default) using the last 100 responses from a node. How long

Re: Read time get worse during dynamic snitch reset

2011-04-12 Thread aaron morton
Something feels odd. From Peters nice write up of the dynamic snitch http://www.mail-archive.com/user@cassandra.apache.org/msg12092.html The RackInferringSnitch (and the PropertyFileSnitch) derive from the AbstractNetworkTopologySnitch and should... In the case of the

Read time get worse during dynamic snitch reset

2011-04-11 Thread shimi
I finally upgraded 0.6.x to 0.7.4. The nodes are running with the new version for several days across 2 data centers. I noticed that the read time in some of the nodes increase by x50-60 every ten minutes. There was no indication in the logs for something that happen at the same time. The only

Re: Read time get worse during dynamic snitch reset

2011-04-11 Thread aaron morton
The reset interval clears the latency tracked for each node so a bad node will be read from again. The scores for each node are then updated every 100ms (default) using the last 100 responses from a node. How long does the bad performance last for? What CL are you reading at ? At Quorum with