Re: Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-09 Thread Nate McCall
Thanks for reporting back, Tom. Can you drop a comment on the ticket with a sentence or two describing your specific case and that speculative_retry = NONE was a valid work-around? That will make it easier for the next folks that come along to have a concrete problem/solution in a single comment

Re: Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-08 Thread Bryan Cheng
Tom, I don't believe so; it seems the symptom would be an indefinite (or very long) hang. To clarify, is this issue restricted to LOCAL_QUORUM? Can you issue a LOCAL_ONE SELECT and retrieve the expected data back? On Tue, Sep 8, 2015 at 12:02 PM, Tom van den Berge < tom.vandenbe...@gmail.com>

Re: Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-08 Thread Tom van den Berge
Just to be sure: can this bug result in a 0-row result while it should be > 0 ? Op 8 sep. 2015 6:29 PM schreef "Tyler Hobbs" : > See https://issues.apache.org/jira/browse/CASSANDRA-9753 > > On Tue, Sep 8, 2015 at 10:22 AM, Tom van den Berge < > tom.vandenbe...@gmail.com>

Re: Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-08 Thread Nate McCall
> > Just to be sure: can this bug result in a 0-row result while it should be > > 0 ? > Per Tyler's reference to CASSANDRA-9753 , you would see this if the read was routed by speculative retry to the nodes that were not yet finished being

Re: Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-08 Thread Tom van den Berge
Nate, I've disabled it, and it's been running for about an hour now without problems, while before, the problem occurred roughly every few minutes. I guess it's safe to say that this proves that CASSANDRA-9753 is the cause of the problem.

Re: Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-08 Thread Tyler Hobbs
See https://issues.apache.org/jira/browse/CASSANDRA-9753 On Tue, Sep 8, 2015 at 10:22 AM, Tom van den Berge < tom.vandenbe...@gmail.com> wrote: > I've been bugging you a few times, but now I've got trace data for a query > with LOCAL_QUORUM that is being sent to a remove data center. > > The

Trace evidence for LOCAL_QUORUM ending up in remote DC

2015-09-08 Thread Tom van den Berge
I've been bugging you a few times, but now I've got trace data for a query with LOCAL_QUORUM that is being sent to a remove data center. The setup is as follows: NetworkTopologyStrategy: {"DC1":"1","DC2":"2"} Both DC1 and DC2 have 2 nodes. In DC2, one node is currently being rebuilt, and