Re: Doubt regarding consistency-level in Cassandra-2.1.10
Glad you got it figured out, but I'm confused about the @Version annotation. The DataStax Java Driver just handles statements, as far as I know it's never going to modify statement text. It sounds like you're using an entity mapping framework on top of the java driver, which uses @Version for optimistic locking, and that upgraded the generated statement to a CAS operation. On Wed, Nov 4, 2015 at 1:20 AM Ajay Gargwrote: > Hi All. > > I think we got the root-cause. > > One of the fields in one of the class was marked with "@Version" > annotation, which was causing the Cassandra-Java-Driver to insert "If Not > Exists" in the insert query, thus invoking SERIAL consistency-level. > > We removed the annotation (didn't really need that), and we have not > observed the error since about an hour or so. > > > Thanks Eric and Bryan for the help !!! > > > Thanks and Regards, > Ajay > > On Wed, Nov 4, 2015 at 8:51 AM, Ajay Garg wrote: > >> Hmm... ok. >> >> Ideally, we require :: >> >> a) >> The intra-DC-node-syncing takes place at the statement/query level. >> >> b) >> The inter-DC-node-syncing takes place at cassandra level. >> >> >> That way, we don't spend too much delay at the statement/query level. >> >> >> For the so-called CAS/lightweight transactions, the above are impossible >> then? >> >> On Wed, Nov 4, 2015 at 5:58 AM, Bryan Cheng >> wrote: >> >>> What Eric means is that SERIAL consistency is a special type of >>> consistency that is only invoked for a subset of operations: those that use >>> CAS/lightweight transactions, for example "IF NOT EXISTS" queries. >>> >>> The differences between CAS operations and standard operations are >>> significant and there are large repercussions for tunable consistency. The >>> amount of time such an operation takes is greatly increased as well; you >>> may need to increase your internal node-to-node timeouts . >>> >>> On Mon, Nov 2, 2015 at 8:01 PM, Ajay Garg >>> wrote: >>> Hi Eric, I am sorry, but I don't understand. If there had been some issue in the configuration, then the consistency-issue would be seen everytime (I guess). As of now, the error is seen sometimes (probably 30% of times). On Mon, Nov 2, 2015 at 10:24 PM, Eric Stevens wrote: > Serial consistency gets invoked at the protocol level when doing > lightweight transactions such as CAS operations. If you're expecting that > your topology is RF=2, N=2, it seems like some keyspace has RF=3, and so > there aren't enough nodes available to satisfy serial consistency. > > See > http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html > > On Mon, Nov 2, 2015 at 1:29 AM Ajay Garg > wrote: > >> Hi All. >> >> I have a 2*2 Network-Topology Replication setup, and I run my >> application via DataStax-driver. >> >> I frequently get the errors of type :: >> *Cassandra timeout during write query at consistency SERIAL (3 >> replica were required but only 0 acknowledged the write)* >> >> I have already tried passing a "write-options with LOCAL_QUORUM >> consistency-level" in all create/save statements, but I still get this >> error. >> >> Does something else need to be changed in >> /etc/cassandra/cassandra.yaml too? >> Or may be some another place? >> >> >> -- >> Regards, >> Ajay >> > -- Regards, Ajay >>> >>> >> >> >> -- >> Regards, >> Ajay >> > > > > -- > Regards, > Ajay >
Re: Doubt regarding consistency-level in Cassandra-2.1.10
Hi All. I think we got the root-cause. One of the fields in one of the class was marked with "@Version" annotation, which was causing the Cassandra-Java-Driver to insert "If Not Exists" in the insert query, thus invoking SERIAL consistency-level. We removed the annotation (didn't really need that), and we have not observed the error since about an hour or so. Thanks Eric and Bryan for the help !!! Thanks and Regards, Ajay On Wed, Nov 4, 2015 at 8:51 AM, Ajay Gargwrote: > Hmm... ok. > > Ideally, we require :: > > a) > The intra-DC-node-syncing takes place at the statement/query level. > > b) > The inter-DC-node-syncing takes place at cassandra level. > > > That way, we don't spend too much delay at the statement/query level. > > > For the so-called CAS/lightweight transactions, the above are impossible > then? > > On Wed, Nov 4, 2015 at 5:58 AM, Bryan Cheng wrote: > >> What Eric means is that SERIAL consistency is a special type of >> consistency that is only invoked for a subset of operations: those that use >> CAS/lightweight transactions, for example "IF NOT EXISTS" queries. >> >> The differences between CAS operations and standard operations are >> significant and there are large repercussions for tunable consistency. The >> amount of time such an operation takes is greatly increased as well; you >> may need to increase your internal node-to-node timeouts . >> >> On Mon, Nov 2, 2015 at 8:01 PM, Ajay Garg wrote: >> >>> Hi Eric, >>> >>> I am sorry, but I don't understand. >>> >>> If there had been some issue in the configuration, then the >>> consistency-issue would be seen everytime (I guess). >>> As of now, the error is seen sometimes (probably 30% of times). >>> >>> On Mon, Nov 2, 2015 at 10:24 PM, Eric Stevens wrote: >>> Serial consistency gets invoked at the protocol level when doing lightweight transactions such as CAS operations. If you're expecting that your topology is RF=2, N=2, it seems like some keyspace has RF=3, and so there aren't enough nodes available to satisfy serial consistency. See http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html On Mon, Nov 2, 2015 at 1:29 AM Ajay Garg wrote: > Hi All. > > I have a 2*2 Network-Topology Replication setup, and I run my > application via DataStax-driver. > > I frequently get the errors of type :: > *Cassandra timeout during write query at consistency SERIAL (3 replica > were required but only 0 acknowledged the write)* > > I have already tried passing a "write-options with LOCAL_QUORUM > consistency-level" in all create/save statements, but I still get this > error. > > Does something else need to be changed in > /etc/cassandra/cassandra.yaml too? > Or may be some another place? > > > -- > Regards, > Ajay > >>> >>> >>> -- >>> Regards, >>> Ajay >>> >> >> > > > -- > Regards, > Ajay > -- Regards, Ajay
Re: Doubt regarding consistency-level in Cassandra-2.1.10
What Eric means is that SERIAL consistency is a special type of consistency that is only invoked for a subset of operations: those that use CAS/lightweight transactions, for example "IF NOT EXISTS" queries. The differences between CAS operations and standard operations are significant and there are large repercussions for tunable consistency. The amount of time such an operation takes is greatly increased as well; you may need to increase your internal node-to-node timeouts . On Mon, Nov 2, 2015 at 8:01 PM, Ajay Gargwrote: > Hi Eric, > > I am sorry, but I don't understand. > > If there had been some issue in the configuration, then the > consistency-issue would be seen everytime (I guess). > As of now, the error is seen sometimes (probably 30% of times). > > On Mon, Nov 2, 2015 at 10:24 PM, Eric Stevens wrote: > >> Serial consistency gets invoked at the protocol level when doing >> lightweight transactions such as CAS operations. If you're expecting that >> your topology is RF=2, N=2, it seems like some keyspace has RF=3, and so >> there aren't enough nodes available to satisfy serial consistency. >> >> See >> http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html >> >> On Mon, Nov 2, 2015 at 1:29 AM Ajay Garg wrote: >> >>> Hi All. >>> >>> I have a 2*2 Network-Topology Replication setup, and I run my >>> application via DataStax-driver. >>> >>> I frequently get the errors of type :: >>> *Cassandra timeout during write query at consistency SERIAL (3 replica >>> were required but only 0 acknowledged the write)* >>> >>> I have already tried passing a "write-options with LOCAL_QUORUM >>> consistency-level" in all create/save statements, but I still get this >>> error. >>> >>> Does something else need to be changed in /etc/cassandra/cassandra.yaml >>> too? >>> Or may be some another place? >>> >>> >>> -- >>> Regards, >>> Ajay >>> >> > > > -- > Regards, > Ajay >
Re: Doubt regarding consistency-level in Cassandra-2.1.10
Hmm... ok. Ideally, we require :: a) The intra-DC-node-syncing takes place at the statement/query level. b) The inter-DC-node-syncing takes place at cassandra level. That way, we don't spend too much delay at the statement/query level. For the so-called CAS/lightweight transactions, the above are impossible then? On Wed, Nov 4, 2015 at 5:58 AM, Bryan Chengwrote: > What Eric means is that SERIAL consistency is a special type of > consistency that is only invoked for a subset of operations: those that use > CAS/lightweight transactions, for example "IF NOT EXISTS" queries. > > The differences between CAS operations and standard operations are > significant and there are large repercussions for tunable consistency. The > amount of time such an operation takes is greatly increased as well; you > may need to increase your internal node-to-node timeouts . > > On Mon, Nov 2, 2015 at 8:01 PM, Ajay Garg wrote: > >> Hi Eric, >> >> I am sorry, but I don't understand. >> >> If there had been some issue in the configuration, then the >> consistency-issue would be seen everytime (I guess). >> As of now, the error is seen sometimes (probably 30% of times). >> >> On Mon, Nov 2, 2015 at 10:24 PM, Eric Stevens wrote: >> >>> Serial consistency gets invoked at the protocol level when doing >>> lightweight transactions such as CAS operations. If you're expecting that >>> your topology is RF=2, N=2, it seems like some keyspace has RF=3, and so >>> there aren't enough nodes available to satisfy serial consistency. >>> >>> See >>> http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html >>> >>> On Mon, Nov 2, 2015 at 1:29 AM Ajay Garg wrote: >>> Hi All. I have a 2*2 Network-Topology Replication setup, and I run my application via DataStax-driver. I frequently get the errors of type :: *Cassandra timeout during write query at consistency SERIAL (3 replica were required but only 0 acknowledged the write)* I have already tried passing a "write-options with LOCAL_QUORUM consistency-level" in all create/save statements, but I still get this error. Does something else need to be changed in /etc/cassandra/cassandra.yaml too? Or may be some another place? -- Regards, Ajay >>> >> >> >> -- >> Regards, >> Ajay >> > > -- Regards, Ajay
Doubt regarding consistency-level in Cassandra-2.1.10
Hi All. I have a 2*2 Network-Topology Replication setup, and I run my application via DataStax-driver. I frequently get the errors of type :: *Cassandra timeout during write query at consistency SERIAL (3 replica were required but only 0 acknowledged the write)* I have already tried passing a "write-options with LOCAL_QUORUM consistency-level" in all create/save statements, but I still get this error. Does something else need to be changed in /etc/cassandra/cassandra.yaml too? Or may be some another place? -- Regards, Ajay
Re: Doubt regarding consistency-level in Cassandra-2.1.10
Serial consistency gets invoked at the protocol level when doing lightweight transactions such as CAS operations. If you're expecting that your topology is RF=2, N=2, it seems like some keyspace has RF=3, and so there aren't enough nodes available to satisfy serial consistency. See http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html On Mon, Nov 2, 2015 at 1:29 AM Ajay Gargwrote: > Hi All. > > I have a 2*2 Network-Topology Replication setup, and I run my application > via DataStax-driver. > > I frequently get the errors of type :: > *Cassandra timeout during write query at consistency SERIAL (3 replica > were required but only 0 acknowledged the write)* > > I have already tried passing a "write-options with LOCAL_QUORUM > consistency-level" in all create/save statements, but I still get this > error. > > Does something else need to be changed in /etc/cassandra/cassandra.yaml > too? > Or may be some another place? > > > -- > Regards, > Ajay >
Re: Doubt regarding consistency-level in Cassandra-2.1.10
Hi Eric, I am sorry, but I don't understand. If there had been some issue in the configuration, then the consistency-issue would be seen everytime (I guess). As of now, the error is seen sometimes (probably 30% of times). On Mon, Nov 2, 2015 at 10:24 PM, Eric Stevenswrote: > Serial consistency gets invoked at the protocol level when doing > lightweight transactions such as CAS operations. If you're expecting that > your topology is RF=2, N=2, it seems like some keyspace has RF=3, and so > there aren't enough nodes available to satisfy serial consistency. > > See > http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html > > On Mon, Nov 2, 2015 at 1:29 AM Ajay Garg wrote: > >> Hi All. >> >> I have a 2*2 Network-Topology Replication setup, and I run my application >> via DataStax-driver. >> >> I frequently get the errors of type :: >> *Cassandra timeout during write query at consistency SERIAL (3 replica >> were required but only 0 acknowledged the write)* >> >> I have already tried passing a "write-options with LOCAL_QUORUM >> consistency-level" in all create/save statements, but I still get this >> error. >> >> Does something else need to be changed in /etc/cassandra/cassandra.yaml >> too? >> Or may be some another place? >> >> >> -- >> Regards, >> Ajay >> > -- Regards, Ajay