[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465389#comment-13465389 ] Sylvain Lebresne commented on CASSANDRA-4545: - bq. Should COUNTER BATCH reject regular row mutations? I don't see why it should Agreed, it should reject them. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465088#comment-13465088 ] Jonathan Ellis commented on CASSANDRA-4545: --- IMO it should reject them, since there is no reason to allow them. :) > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465074#comment-13465074 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- Right. Still don't want to do that. But I'm outnumbered here. All right. So BEGIN COUNTER BATCH it is, an option that allows counters. Should COUNTER BATCH reject regular row mutations? I don't see why it should, but need your opinion on this. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465070#comment-13465070 ] Jonathan Ellis commented on CASSANDRA-4545: --- To clarify: moving people away from Thrift is a Good Thing, so my position is we should support batched counters so cql3 is a realistic option for counter users. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465067#comment-13465067 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- Yes, batch_mutate still allows counter mutations and that shouldn't be changed. atomic_batch_mutate rejected them since day one. If someone really needs that performance then there is always thrift (and for now - cql2, too). And cql can be used for everything else. I doubt counter mutations in batches are used by many people anyway, especially with cql. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465062#comment-13465062 ] Jonathan Ellis commented on CASSANDRA-4545: --- I'll buy that we should do it for performance, or we force counter users to stay on Thrift. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465059#comment-13465059 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- If we end up allowing counter batches, which I don't want to do at all, then we should at least require explicit BEGIN COUNTER BATCH statement, this is where I agree with Sylvain. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465018#comment-13465018 ] Sylvain Lebresne commented on CASSANDRA-4545: - bq. Why is that? Can't I write UPDATE foo SET x = x + 1, y = y + 2, ... ? Sure you can, but that only works for columns in the same "CQL" row. If you have a wide row (a.k.a a composite PK), you can't update multiple ones without a BATCH (and for *the* good use case of counters, realtime analytics, doing time series of counter is really useful). And without a BATCH, you do lose the atomicity and isolation we currently have in that case (on top of the performance penalty). bq. to give people a gun to shoot their feet with I'm not sure I understand why that's so much a "gun to shoot their feet with". As far as I can tell, allowing counter batches adds no foot shooting over allowing counters (which have by themsleves some foot shooting involved, but that's another problem). You should not replay counter inserts, but whether they are in batch or not don't make a whole of a difference (in both case you don't know what insert went in or not). > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[
https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464964#comment-13464964
]
Jonathan Ellis commented on CASSANDRA-4545:
---
bq. even for inserts within the same row key you will have to use BATCH most of
the time
Why is that? Can't I write {{UPDATE foo SET x = x + 1, y = y + 2, ...}} ?
> add cql support for batchlog
>
>
> Key: CASSANDRA-4545
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4545
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jonathan Ellis
>Assignee: Aleksey Yeschenko
> Attachments: CASSANDRA-4545.txt
>
>
> Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464961#comment-13464961 ] Sylvain Lebresne commented on CASSANDRA-4545: - bq. Is the overhead saved by BATCH large enough that we care about allowing counters in BATCHes? There is no doubt it is. Again, keep in mind that even for inserts within the same row key you will have to use BATCH most of the time. In that case, it's clear shoving all insert inside the same IMutation will be way faster than doing back and forth with the client. That's even more true for counter where each of those back and forth would include a local read (that is also "batched" if everything is in the same mutation). > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464935#comment-13464935 ] Jonathan Ellis commented on CASSANDRA-4545: --- Is the overhead saved by BATCH large enough that we care about allowing counters in BATCHes? since they can't be either atomic or replayable the only reason left to allow batch is performance, and I'm not sure that's a good enough reason to give people a gun to shoot their feet with, particularly now that we have prepared statements. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464921#comment-13464921 ] Sylvain Lebresne commented on CASSANDRA-4545: - I'll reformulate: we don't want to completely refuse counters, at least not in unlogged batches. Sure batch of counters shouldn't be replay, but that's not a good reason to refuse batches in the first place. Typically CASSANDRA-4723 exists exactly so user can know it was a counter batch and should not be replay. Or to be more precise, if you timeout with counters you're screwed, them being in batch or not, so allowing batch of counters is not worst than allowing counters in the first place, so why forbid them? Especially since with CQL3, you often have to use a BATCH even to insert data in the same "Cassandra" row, and so disallowing BATCH of counters would be a major regression compared to what you can do with thrift. It's true though that we need to decide if we allow the BEGIN BATCH for counters, or only the BEGIN UNLOGGED BATCH one, or both. We could even say that counter batch needs to use a specific syntax like BEGIN COUNTER BATCH. Given that counter are a specific case anyway as far as failed write is concerned, I think just allowing BEGIN BATCH would probably not be a big deal, though I do kind of like the idea of a BEGIN COUNTER BATCH syntax as it highlight the fact that counter and regular column cannot be mixed. I don't care too much one way or the other. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464898#comment-13464898 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- I'm pretty sure we DO want to forbid counter mutations in batches entirely. Counter mutations are not idempotent and therefore not safe to replay. As for rebasing the patch - will do. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464852#comment-13464852 ] Sylvain Lebresne commented on CASSANDRA-4545: - Unfortunately that patch already need rebase. Also, we don't want to completely refuse counters in batches, we want to refuse batches that mix counters and regular inserts. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Attachments: CASSANDRA-4545.txt > > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462084#comment-13462084 ] Sylvain Lebresne commented on CASSANDRA-4545: - bq. What's your position on making abm default and leaving old (non-atomic) behaviour to BEGIN UNLOGGED BATCH? I'm good with that. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462072#comment-13462072 ] Sylvain Lebresne commented on CASSANDRA-4545: - That being said, even if you don't allow to mix regular and counter, you still have the fact that BEGIN BATCH will be atomic for regular but not for counters. But I personally can live with that since counters already differ from regular inserts when it comes to failed writes. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462073#comment-13462073 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- Yep, that's the easier decision. What's your position on making abm default and leaving old (non-atomic) behaviour to BEGIN UNLOGGED BATCH? > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462067#comment-13462067 ] Sylvain Lebresne commented on CASSANDRA-4545: - I agree with Jonathan. Allowing counter and other in the same batch is a bad idea and it's only allowed because we forgot to disallow it. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462051#comment-13462051 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- bq. Still think the right thing to do is disallow counters from batches entirely, at least for CQL3. Ok. Will do that as part of this issue then. Sylvain, what do you think? I need a final decision here, to get it over with (and 4636 after that). > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462037#comment-13462037 ] Jonathan Ellis commented on CASSANDRA-4545: --- bq. Allowing counter mutations in regular batches and not allowing them in atomic batches still might be a problem Still think the right thing to do is disallow counters from batches entirely, at least for CQL3. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462019#comment-13462019 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- I think I agree with Jonathan. Correctness is more important than speed, especially if you give users the fast option as well. Allowing counter mutations in regular batches and not allowing them in atomic batches still might be a problem, though - this will break some code, not just make it slower. Otherwise I'm in favour of mapping BEGIN BATCH to atomic_batch_mutate and BEGIN UNLOGGED BATCH to batch_mutate. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[
https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461874#comment-13461874
]
Jonathan Ellis commented on CASSANDRA-4545:
---
I think I would favor {{BEGIN BATCH}} for atomic and {{BEGIN UNLOGGED BATCH}}
otherwise. Reasoning:
# I'm pessimistic that people will (a) understand the {{ATOMIC|NONATOMIC}}
distinction without (b) being needlessly scared. Doing the "right" thing by
default means we don't have to inflict a decision on new users that they're not
yet qualified to make.
# Secondarily, this is a gentler "breaking" of your code (for hypothetical cql3
1.1 users) vs making {{BEGIN BATCH}} not work without a qualifier. If someone
upgrading from cql3 in 1.1 notices the slowdown (or reads NEWS) they can switch
to {{UNLOGGED}} if they want. Granted, we explicitly said cql3 in 1.1 is beta,
but if we can avoid breaking things, so much the better.
I don't have a good answer for the "someone could compare Thrift
{{batch_mutate}} with 1.2 {{BEGIN BATCH}} without noticing that it really
corresponds to {{atomic_batch_mutate}}" concern, other than that I'm willing to
live with that price.
> add cql support for batchlog
>
>
> Key: CASSANDRA-4545
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4545
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jonathan Ellis
>Assignee: Aleksey Yeschenko
>
> Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461812#comment-13461812 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- No options looks ideal atm, so here is another non-ideal one: leave current syntax as is and add BEGIN FULLY-ATOMIC BATCH (or COMPLETELY ATOMIC) for abm, not just ATOMIC, since regular batches have per-row atomicity. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461803#comment-13461803 ] Sylvain Lebresne commented on CASSANDRA-4545: - Ok, so atomic batches are noticeably slower than non atomic ones. Can't say it's a big surprise but it's nice to some numbers at least. Tbh I don't have a strong conviction on what is the best solution here. On the one side, I like the idea of having all our write operations being atomic by default and I think ideally they should be. On the other side, I know that if we use atomic by default, since it's slower, a fair amount of people that upgrade to CQL3 will conclude that "CQL3 is slower than thrift" without looking further. And I wouldn't totally blame them. In that context I don't dislike the option of making the choice explicit in the syntax (i.e. allowing 'BEGIN ATOMIC BATCH' and 'BEGIN NON-ATOMIC BATCH' but not 'BEGIN BATCH'), but that would be one more syntax breaking change in CQL3 we make fairly late before the release, which is not ideal. Might still be my slightly prefered choice at this point, but again, not a very strong conviction. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461582#comment-13461582 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- Used a three-node cluster, rah modified cassandra-stress from a separate machine. cassandra-stress was modified to take keys-per-batch parameter. I wiped all the data, ran stress for regular batch mutate (three times for each kpb value, picking the average one), then wiped all the data again and ran the same benchmarks for atomic batch mutate. "total" in the output is total batches, not total keys (unlike in unmodified cassandra-stress). Default total number of keys was written every time (1M) with 5 columns per key and CL.ONE. # 1 key per batch ## batch_mutate 264131,26413,26413,0.001746815027391711,10 612819,34868,34868,0.0013791584453723674,20 988733,37591,37591,0.001265645865809733,30 100,1126,1126,9.46924647199787E-4,30 ## atomic_batch_mutate 116329,11632,11632,0.0039553851576133205,10 218728,10239,10239,0.0049213175909921,20 341348,12262,12262,0.004092179089871147,30 453569,11222,11222,0.00446199909107921,40 553009,9944,9944,0.005039782783588093,50 657635,10462,10462,0.004790099975149581,60 771908,11427,11427,0.004257716170924015,70 872911,10100,10100,0.005103590982446066,80 970814,9790,9790,0.003992002287978918,90 100,2918,2918,0.00181138216958,93 # 10 keys per batch ## batch_mutate 41456,4145,4145,0.009599430721729063,10 95977,5452,5452,0.00859437647878799,20 10,402,402,0.005297787720606513,21 ## atomic_batch_mutate 31813,3181,3181,0.01379417219375727,10 61825,3001,3001,0.017426962548314006,20 92392,3056,3056,0.013992279255406156,30 10,760,760,0.018285094637223973,34 # 100 keys per batch ## batch_mutate 5414,541,541,0.07286830439601034,10 1,458,458,0.06448626253815962,16 ## atomic_batch_mutate 4560,456,456,0.077079167,10 9037,447,447,0.10926133571588117,20 1,96,96,0.04469574247144341,22 # 1000 keys per batch ## batch_mutate 537,53,53,0.6769962756052141,10 1000,46,46,0.5918012958963282,16 ## atomic_batch_mutate 509,50,50,0.6374538310412574,10 1000,49,49,0.9193156822810591,20 # 1 keys per batch ## batch_mutate 40,4,4,7.91995,19 100,6,6,8.1601832,30 ## atomic_batch_mutate 17,1,1,6.496764705882353,19 36,1,1,11.307736842105264,29 98,6,6,15.237580645161291,40 100,0,0,2.745,40 I don't have an opinion yet regarding making abm the default batch mode, but these are some numbers. Please let me know if you need more (and what kind of scenarios). > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459197#comment-13459197 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- I'm going to add more numbers on the weekends - with batches including mutations to different rows. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459174#comment-13459174 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- I've done some simple benchmarking using cassandra-stress Inserter and modified Inserter that uses atomic_batch_mutate instead of batch_mutate. I used a three-node cluster, wiped and restarted every machine between test runs. cassandra-stres was being run from a fourth machine. Every batch only contains mutations to one partition key. I'll have to modify Stress more to include mutations of different rows in a batch. The difference will be larger there (regular mutations will be sent to different machines in parallel, but the whole serialized batch itself will still be sent in one piece, and we'll be waiting for at least one response (CL.ONE) before distributing regular mutations). So take the following numbers as the least possible difference between batch_mutate and atomic_batch_mutate or atomic_batch_mutate best case scenario. -n 1000 -c 1 == batch mutate 1000,100,100,0.008556,0 == atomic batch mutate 1000,100,100,0.024504,3 -n 1000 -c 10 == batch mutate 1000,100,100,0.010149,0 == atomic batch mutate 1000,100,100,0.032331,1 -n 1000 -c 100 == batch mutate 1000,100,100,0.038674,2 == atomic batch mutate 1000,100,100,0.041967,2 -n 1000 -c 1000 == batch mutate 1000,100,100,0.088965,4 == atomic batch mutate 1000,100,100,0.136551,6 -n 1000 -c 1 == batch mutate 547,54,54,0.6684040219378428,11 1000,45,45,0.5687505518763797,18 == atomic batch mutate 546,54,54,0.7630347985347985,11 924,37,37,0.768542328042328,21 1000,7,7,1.5153157894736842,24 -n 50 -c 10 == batch mutate 8,0,0,8.057,23 33,2,2,14.80572,33 50,1,1,18.833529411764705,35 == atomic batch mutate 10,1,1,6.5336,20 30,2,2,14.04065,30 40,1,1,27.3045,40 50,1,1,31.8171,46 > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457257#comment-13457257 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- bq. we allow counters in normal batches? that's a bug waiting to happen since you can't replay a counter update safely... We do. At least in thrift API and cql3, most likely in cql2, too. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457252#comment-13457252 ] Jonathan Ellis commented on CASSANDRA-4545: --- we allow counters in normal batches? that's a bug waiting to happen since you can't replay a counter update safely... > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457239#comment-13457239 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- Another thing to remember is that counter mutations are not allowed in atomic batches, so the two statements won't be functionally same anyway. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457049#comment-13457049 ] Sylvain Lebresne commented on CASSANDRA-4545: - bq. maybe it's simpler to just say "clients should be responsible for this optimization" in the first place I disagree. I don't want user to be like "Hum, I know I need atomicity for that query but wait ... I'm only writing to one partition key so I'd better not use the ATOMIC keyword in my query because that's slower and those lazy C* devs didn't bother adding a one line 'if' to optimize the atomic path. Oh, and wait, I'd better add a comment that this needs to be atomic since someone else reading my query would think it's not required". Besides, the details of when to retry should hopefully largely be hidden by the client library, but that optimization cannot, or at least not easily as this would require parsing the query. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457027#comment-13457027 ] Jonathan Ellis commented on CASSANDRA-4545: --- bq. it would be nice to make "batchlog" writes skip the batchlog patch if the batch is on only one partition key true, although that does mean clients need to be aware of the difference as well (clients need to retry if it timesout during the batchlog write, or if it times out during the data write and there is only one key), so maybe it's simpler to just say "clients should be responsible for this optimization" in the first place. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456932#comment-13456932 ] Sylvain Lebresne commented on CASSANDRA-4545: - As a side note let me remark that a BATCH is not synonymous of "non atomic", since everything within the same partition key will be atomic and isolated (and in a fair amount of cases, those can be done with a single INSERT/UPDATE). It would be nice if whatever choice of syntax we make here don't suggest too much that what is atomic is not. And as a side side note, it would be nice to make "batchlog" writes skip the batchlog patch if the batch is on only one partition key. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455118#comment-13455118 ] Jonathan Ellis commented on CASSANDRA-4545: --- bq. What would be really nice though to make that decision would be to have a few benchmarks to estimate how much of an impact it is exactly. Let's start with that, then. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455102#comment-13455102 ] Sylvain Lebresne commented on CASSANDRA-4545: - I'm not fond of having a yaml setting for that, it over-complicate things. I do note however that we could decide to make the choice explicit, i.e. removing 'BEGIN BATCH' altogether (again if we target 1.2, we can do that) and only supporting either 'BEGIN ATOMIC BATCH' or 'BEGIN NON-ATOMIC BATCH'. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455091#comment-13455091 ] Aleksey Yeschenko commented on CASSANDRA-4545: -- We can create two versions - BEGIN .. APPLY ATOMICALLY and BEGIN .. APPLY NON-ATOMICALLY (or BEGIN NON-ATOMIC BATCH and BEGIN ATOMIC BATCH), or some better names. This comment is not about names. And leave the BEGIN .. APPLY syntax. Make a setting in cassandra.yaml that would determine how to treat BEGIN .. BATCH statements - as atomic or non-atomic. And make atomic the default. Upgraders without the setting will get the old behaviour - no atomicity, no unpleasant surprises. New users will get atomicity by default, which, I agree, is a better default. Having two versions of the statement will also allow users let users to choose on per-query basis, without changing any settings. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[
https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455067#comment-13455067
]
Sylvain Lebresne commented on CASSANDRA-4545:
-
What would be really nice though to make that decision would be to have a few
benchmarks to estimate how much of an impact it is exactly.
And on the one side, if we ship that for 1.2, CQL3 is basically new (as in, it
was beta in 1.1), so if we want to make it the default, we should do it now or
never.
I guess in an ideal world I'd prefer having the safer option be the default,
but I can agree with the "unpleasant surprise" argument and I'd probably be
fine with a syntax like:
{noformat}
BEGIN ATOMIC BATCH
{noformat}
to use batchlog. The only slight downside is that this could lead people to
believe that if they want a single insert to be atomic, they might need to put
it in an atomic batch.
> add cql support for batchlog
>
>
> Key: CASSANDRA-4545
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4545
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jonathan Ellis
>Assignee: Aleksey Yeschenko
>
> Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4545) add cql support for batchlog
[ https://issues.apache.org/jira/browse/CASSANDRA-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434263#comment-13434263 ] Jonathan Ellis commented on CASSANDRA-4545: --- Sylvain proposed making this the default, which I'm not convinced of -- I suspect the performance impact will not be negligible, so this could be an unpleasant surprise for upgraders. Either way though we need syntax to request batchlog either on or off, depending on which we do make the default. > add cql support for batchlog > > > Key: CASSANDRA-4545 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4545 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis > > Need to expose the equivalent of atomic_batch_mutate (CASSANDRA-4542) to CQL3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
