[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[
https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663220#comment-13663220
]
Jonathan Ellis commented on CASSANDRA-5314:
---
Why do we add this?
{code}
cf.addColumn(new Column(columnName(),
ByteBufferUtil.EMPTY_BYTE_BUFFER, timestamp));
{code}
Nit: CopyOnWriteArraySet is a bit of an odd choice, since we expect to mutate
it once for each entry.
Replaying old batches can 'undo' deletes
Key: CASSANDRA-5314
URL: https://issues.apache.org/jira/browse/CASSANDRA-5314
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 1.2.6
Batchlog manager does not subtract the time spent in the batchlog from hints'
ttls and this may cause undoing deletes. The attached patch fixes it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663245#comment-13663245 ] Aleksey Yeschenko commented on CASSANDRA-5314: -- bq. Why do we add this? It's the CQL3 row marker. Technically we don't have to do this, but it's just the right way to do it and I would sleep better with it in place. Could prevent something like CASSANDRA-5572 from happening again in the future (although it's true that we only remove the whole rows in the batchlog so the probability of something like this is low). bq. Nit: CopyOnWriteArraySet is a bit of an odd choice, since we expect to mutate it once for each entry. It's mostly irrelevant here - just used the first thread-safe set that came to mind. Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 1.2.6 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663255#comment-13663255 ] Jonathan Ellis commented on CASSANDRA-5314: --- +1 Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 1.2.6 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13650930#comment-13650930 ] Aleksey Yeschenko commented on CASSANDRA-5314: -- So, for now: 1. Apply the original patch (subtract the time spent in the batchlog from hints' ttls) 2. If the target nodes are up (of at least FD says so), try writing the mutations directly instead of hinting them 3. Reduce the batch replay interval to, say, 1 minute 4. Fix hints to not write the hint at all if it's computed TTL == 0 (when gc_grace is 0, for example). Am I missing anything? Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 2.0 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13600936#comment-13600936 ] Sylvain Lebresne commented on CASSANDRA-5314: - bq. We TTL hints with gcgs to prevent this Interesting. There is 2 (somewhat conflicting) things that seems weird to me with that: # the intent seems broken if gc_grace==0, since a TTL of 0 means no TTL at all. # in the (not uncommon) case of using TTL on all columns without doing manual deletion, it makes sense to use a very low gc_grace. In this case, this seems to make hints useless, which I'm not sure is what people expect (I'll admit I was unaware of that). Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.2 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 1.2.3 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13600274#comment-13600274 ] Sylvain Lebresne commented on CASSANDRA-5314: - Isn't that a problem we already have with normal hints? After all, if an insert takes a long time to get delivered and you have a short gc_grace, some delete that override an hint could get gced before the hint gets delivered. In any case, I agree with Jonathan: saying that you shouldn't have a gc_grace too short if you do deletes seems fair to me, in the sense that it's what gc_grace is about: providing some time frame after which you consider everything has been delivered. Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.2 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 1.2.3 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13600298#comment-13600298 ] Jonathan Ellis commented on CASSANDRA-5314: --- We TTL hints with gcgs to prevent this, iirc. Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.2 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 1.2.3 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5314) Replaying old batches can 'undo' deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599372#comment-13599372 ] Jonathan Ellis commented on CASSANDRA-5314: --- Hmm, this is tricky: as you say, we don't want to undo more-recent deletes, but we also don't want to discard other rows in the batch, or even the row with low gcgs, that may be necessary to preserve our Atomic Batch guarantees. I.e.: replaying the batch may be incorrect, but not replaying it may also be incorrect. I think the only solution is to require that gcgs be greater than the batch timeout we use. Replaying old batches can 'undo' deletes Key: CASSANDRA-5314 URL: https://issues.apache.org/jira/browse/CASSANDRA-5314 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.2 Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 1.2.3 Batchlog manager does not subtract the time spent in the batchlog from hints' ttls and this may cause undoing deletes. The attached patch fixes it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
