Re: COMMERCIAL:Re: Cross-datacenter requests taking a very long time.

[Andrew Vant]

On Mar 31, 2015, at 4:59 PM, daemeon reiydelle  wrote:
> What is your replication factor?

NetworkTopologyStrategy with replfactor: 2 in each DC. 

Someone else asked about the endpoint snitch I'm using; it's set to 
GossipingPropertyFileSnitch.

> Any idea how much data has to be processed under the query?

It does not matter what query I use, or what size; the problem occurs even just 
selecting a single user from the users table.

> While running the query against both DC's, you can take a look at netstats
> to get a really quick-and-dirty idea of network traffic.

I'll try that. I should add that one of the other teams here has a similar 
setup (3 nodes in 3 DCs) that is working correctly. We're going to go through 
the config files and see if we can figure out what's different. 

-- 

Andrew

Cross-datacenter requests taking a very long time.

[Andrew Vant]

I have a Cassandra 2.0.13 cluster with three datacenters, three nodes per 
datacenter. If I open cqlsh and do a select with any consistency level that 
crosses datacenters (e.g. QUORUM or ALL), it works, but takes 2+ minutes to 
return. The same statement with consistency ONE or LOCAL_QUORUM is as fast as 
it should be. It does not appear to be latency between centers; I can point 
cqlsh at a server in a different DC and it's not noticeably slow.

I tried turning tracing on to get a better idea of what was happening; but it 
complains `Session  wasn't found`. 

I'm not entirely sure what direction to look in to find the problem. 

-- 

Andrew