Re: Counters: consistency, atomic batch
Thanks, good article. But some of my questions are still unanswered. I will reformulate and post them as short separate emails. On 05.09.14 01:01, Ken Hancock wrote: Counters are way more complicated than what you're illustrating. Datastax did a good blog post on this: http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters On Thu, Sep 4, 2014 at 6:34 AM, Eugene Voytitsky viy@gmail.com mailto:viy@gmail.com wrote: Hi all, I am using Cassandra 2.0.x. and Astyanax 1.56.x (2.0.1 shows the same results) driver via Thrift protocol. Questions about counters: 1. Consistency. Consider simplest case when we update value of single counter. 1.1. Is there any difference between updating counter with ONE or QUORUM level? Yes, I understand that ONE may affect reading - readers may see old value. It's ok, eventual consistency for the reader is ok. I am asking, whether writing counter with ONE may lead to totally broken data? I will explain. * Host A stores most recent value 100, host B stores old value 99 (isn't replicated yet). * I increment counter with ONE. Request is sent to host B. * B sees 99. Adds 1. Saves 100, and this 100 bacame more new than old 100 stored on host A. Later it will be replicated to A. * Result: we lost 1 increment, cause actually value should be 101, not 100. As I understand this scenario isn't possible with QUORUM nor ONE. Because actually Cassandra stores counter value in shard structure. So I can safely update counter value with ONE. Am I right? 1.2. If I update counter with QUORUM level whether Cassandra read the old value also with QUORUM level? Or the same point with local shard makes possible to read only value stored on the host which doing writing? 1.3. How 1.1 and 1.2 behavior will change in Cassandra 2.1 and 3.0? I read that in Cassandra 2.1 counters are totally reimplemented. And in 3.0 will be too again. 2. Atomicity. I need to log 1 event as increments to several tables (yes, we use data duplication for different select queries) I use single batch mutation for all increments. Can Cassandra execute batch of counters increments in atomic manner? Here: http://www.datastax.com/dev/__blog/atomic-batches-in-__cassandra-1-2 http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2 I see the following: 1.2 also introduces a separate BEGIN COUNTER BATCH for batched counter updates. Unlike other writes, counter updates are not idempotent, so replaying them automatically from the batchlog is not safe. Counter batches are thus strictly for improved performance when updating multiple counters in the same partition. The text isn't 100% clear. Does it mean that Cassandra can't guarantee atomic batch for counters even with BEGIN COUNTER BATCH? If it can't, in which Cassandra version atomic batch for counters will work? And what is the difference between 'BEGIN COUNTER BATCH' and 'BEGIN BATCH'? If it can, do you know which driver supports BEGIN COUNTER BATCH? I searched the whole source of Astyanax 2.0.1 and it seems that it doesn't support it currently. Thanks in advance! PS. Do you know how to communicate with Astyanax team? I wrote several questions to google groups email astyanax-cassandra-client@__googlegroups.com mailto:astyanax-cassandra-cli...@googlegroups.com but didn't receive any answers. -- Best regards, Eugene Voytitsky -- **Ken Hancock **| System Architect, Advanced Advertising SeaChange International 50 Nagog Park Acton, Massachusetts 01720 ken.hanc...@schange.com mailto:ken.hanc...@schange.com | www.schange.com http://www.schange.com/ | NASDAQ:SEAC http://www.schange.com/en-US/Company/InvestorRelations.aspx Office: +1 (978) 889-3329 | Google Talk: ken.hanc...@schange.com mailto:ken.hanc...@schange.com | Skype:hancockks | Yahoo IM:hancockksLinkedIn http://www.linkedin.com/in/kenhancock SeaChange International http://www.schange.com/ This e-mail and any attachments may contain information which is SeaChange International confidential. The information enclosed is intended only for the addressees herein and may not be copied or forwarded without permission from SeaChange International. -- Best regards, Eugene Voytitsky
Counters: consistency, atomic batch
Hi all, I am using Cassandra 2.0.x. and Astyanax 1.56.x (2.0.1 shows the same results) driver via Thrift protocol. Questions about counters: 1. Consistency. Consider simplest case when we update value of single counter. 1.1. Is there any difference between updating counter with ONE or QUORUM level? Yes, I understand that ONE may affect reading - readers may see old value. It's ok, eventual consistency for the reader is ok. I am asking, whether writing counter with ONE may lead to totally broken data? I will explain. * Host A stores most recent value 100, host B stores old value 99 (isn't replicated yet). * I increment counter with ONE. Request is sent to host B. * B sees 99. Adds 1. Saves 100, and this 100 bacame more new than old 100 stored on host A. Later it will be replicated to A. * Result: we lost 1 increment, cause actually value should be 101, not 100. As I understand this scenario isn't possible with QUORUM nor ONE. Because actually Cassandra stores counter value in shard structure. So I can safely update counter value with ONE. Am I right? 1.2. If I update counter with QUORUM level whether Cassandra read the old value also with QUORUM level? Or the same point with local shard makes possible to read only value stored on the host which doing writing? 1.3. How 1.1 and 1.2 behavior will change in Cassandra 2.1 and 3.0? I read that in Cassandra 2.1 counters are totally reimplemented. And in 3.0 will be too again. 2. Atomicity. I need to log 1 event as increments to several tables (yes, we use data duplication for different select queries) I use single batch mutation for all increments. Can Cassandra execute batch of counters increments in atomic manner? Here: http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2 I see the following: 1.2 also introduces a separate BEGIN COUNTER BATCH for batched counter updates. Unlike other writes, counter updates are not idempotent, so replaying them automatically from the batchlog is not safe. Counter batches are thus strictly for improved performance when updating multiple counters in the same partition. The text isn't 100% clear. Does it mean that Cassandra can't guarantee atomic batch for counters even with BEGIN COUNTER BATCH? If it can't, in which Cassandra version atomic batch for counters will work? And what is the difference between 'BEGIN COUNTER BATCH' and 'BEGIN BATCH'? If it can, do you know which driver supports BEGIN COUNTER BATCH? I searched the whole source of Astyanax 2.0.1 and it seems that it doesn't support it currently. Thanks in advance! PS. Do you know how to communicate with Astyanax team? I wrote several questions to google groups email astyanax-cassandra-cli...@googlegroups.com but didn't receive any answers. -- Best regards, Eugene Voytitsky
Re: Counters: consistency, atomic batch
Counters are way more complicated than what you're illustrating. Datastax did a good blog post on this: http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters On Thu, Sep 4, 2014 at 6:34 AM, Eugene Voytitsky viy@gmail.com wrote: Hi all, I am using Cassandra 2.0.x. and Astyanax 1.56.x (2.0.1 shows the same results) driver via Thrift protocol. Questions about counters: 1. Consistency. Consider simplest case when we update value of single counter. 1.1. Is there any difference between updating counter with ONE or QUORUM level? Yes, I understand that ONE may affect reading - readers may see old value. It's ok, eventual consistency for the reader is ok. I am asking, whether writing counter with ONE may lead to totally broken data? I will explain. * Host A stores most recent value 100, host B stores old value 99 (isn't replicated yet). * I increment counter with ONE. Request is sent to host B. * B sees 99. Adds 1. Saves 100, and this 100 bacame more new than old 100 stored on host A. Later it will be replicated to A. * Result: we lost 1 increment, cause actually value should be 101, not 100. As I understand this scenario isn't possible with QUORUM nor ONE. Because actually Cassandra stores counter value in shard structure. So I can safely update counter value with ONE. Am I right? 1.2. If I update counter with QUORUM level whether Cassandra read the old value also with QUORUM level? Or the same point with local shard makes possible to read only value stored on the host which doing writing? 1.3. How 1.1 and 1.2 behavior will change in Cassandra 2.1 and 3.0? I read that in Cassandra 2.1 counters are totally reimplemented. And in 3.0 will be too again. 2. Atomicity. I need to log 1 event as increments to several tables (yes, we use data duplication for different select queries) I use single batch mutation for all increments. Can Cassandra execute batch of counters increments in atomic manner? Here: http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2 I see the following: 1.2 also introduces a separate BEGIN COUNTER BATCH for batched counter updates. Unlike other writes, counter updates are not idempotent, so replaying them automatically from the batchlog is not safe. Counter batches are thus strictly for improved performance when updating multiple counters in the same partition. The text isn't 100% clear. Does it mean that Cassandra can't guarantee atomic batch for counters even with BEGIN COUNTER BATCH? If it can't, in which Cassandra version atomic batch for counters will work? And what is the difference between 'BEGIN COUNTER BATCH' and 'BEGIN BATCH'? If it can, do you know which driver supports BEGIN COUNTER BATCH? I searched the whole source of Astyanax 2.0.1 and it seems that it doesn't support it currently. Thanks in advance! PS. Do you know how to communicate with Astyanax team? I wrote several questions to google groups email astyanax-cassandra-cli...@googlegroups.com but didn't receive any answers. -- Best regards, Eugene Voytitsky -- *Ken Hancock *| System Architect, Advanced Advertising SeaChange International 50 Nagog Park Acton, Massachusetts 01720 ken.hanc...@schange.com | www.schange.com | NASDAQ:SEAC http://www.schange.com/en-US/Company/InvestorRelations.aspx Office: +1 (978) 889-3329 | [image: Google Talk:] ken.hanc...@schange.com | [image: Skype:]hancockks | [image: Yahoo IM:]hancockks [image: LinkedIn] http://www.linkedin.com/in/kenhancock [image: SeaChange International] http://www.schange.com/This e-mail and any attachments may contain information which is SeaChange International confidential. The information enclosed is intended only for the addressees herein and may not be copied or forwarded without permission from SeaChange International.
