[
https://issues.apache.org/jira/browse/CASSANDRA-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-12811:
------------------------------------
Status: Patch Available (was: Open)
The patch is ready. 3.0 is slightly different, but it's just a single line
[here|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:12811-3.0#diff-2e17efa5977a71330df6651d3bec0d12R739].
The issue was rather difficult to reproduce, since as I mentioned it looks like
a combination of unfortunate events (incidental flush + unmeant range
tombstones), but you can stably reproduce it in the mentioned `DeleteTest` by
making sure `flush` is happening right before [this
delete|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:12811-3.0#diff-70d3a7f61389330811d6eb2f7d2d1b76L443].
To prove that this is in fact the same issue (on CI and reproduced one), here's
an mem/sstable dump from the CI, you can see there that, even though the run
was with {{forceFlush == false}}, the memtable was non-empty. And it happened
right before the delete occurred, so it went to the memtable, whereas all the
data (including the {{value}} column tombstone) went to the sstable.
{code}
Memtables:
INFO [main] 2017-03-06 10:46:20,411 partition.partitionKey() =
DecoratedKey(-3485513579396041028, 00000000)
INFO [main] 2017-03-06 10:46:20,411 row = Marker '@'1488797180382000
INFO [main] 2017-03-06 10:46:20,411 ByteBufferUtil.bytesToHex(buffer) =
00000001
INFO [main] 2017-03-06 10:46:20,411 row = Marker '@'1488797180382000
INFO [main] 2017-03-06 10:46:20,411 ByteBufferUtil.bytesToHex(buffer) =
00000001
SSTables:
INFO [main] 2017-03-06 10:46:20,411 partition.partitionKey() =
DecoratedKey(-4069959284402364209, 00000001)
INFO [main] 2017-03-06 10:46:20,412 row = [[value=6 ts=1488797180328000]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000000
INFO [main] 2017-03-06 10:46:20,412 partition.partitionKey() =
DecoratedKey(-3485513579396041028, 00000000)
INFO [main] 2017-03-06 10:46:20,412 row = [[value=0 ts=1488797180279000]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000000
INFO [main] 2017-03-06 10:46:20,412 row = [[value=<tombstone>
ts=1488797180337000 ldt=1488797180]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000001
INFO [main] 2017-03-06 10:46:20,412 row = [[value=2 ts=1488797180309000]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000002
INFO [main] 2017-03-06 10:46:20,412 row = [[value=3 ts=1488797180313000]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000003
INFO [main] 2017-03-06 10:46:20,412 row = [[value=4 ts=1488797180320000]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000004
INFO [main] 2017-03-06 10:46:20,412 row = [[value=5 ts=1488797180323000]]
INFO [main] 2017-03-06 10:46:20,412 ByteBufferUtil.bytesToHex(buffer) =
00000005
{code}
I've split it into several commits:
* fixing the test itself (as most likely brackets in delete statement were
unintentional)
* writing tests to cover more cases with 2 sstables, mem/sstable, an sstable
and a memtable
* fixing the issue with memtable reads by removing `hasNext` from the
`searchIterator` as it can not reliably report whether or not there is next
item, since it doesn't know what clustering {{next(Clustering clustering)}} is
going to be called with. This is particularly important in cases with
all-rangetombstone partitions.
* fixing one more issue discovered by the test, in cases when we have two
sstables and one of them has only a range tombstone.
|[3.0|https://github.com/ifesdjeen/cassandra/tree/12811-3.0]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12811-3.0-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12811-3.0-dtest/]|
|[3.11|https://github.com/ifesdjeen/cassandra/tree/12811-3.11]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12811-3.11-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12811-3.11-dtest/]|
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12811-preliminary]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12811-preliminary-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12811-preliminary-dtest/]|
> testall failure in
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns-compression
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-12811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12811
> Project: Cassandra
> Issue Type: Bug
> Reporter: Sean McCarthy
> Assignee: Alex Petrov
> Labels: test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.X_testall/34/testReport/org.apache.cassandra.cql3.validation.operations/DeleteTest/testDeleteWithOneClusteringColumns_compression/
> {code}
> Error Message
> Expected empty result but got 1 rows
> {code}
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: Expected empty result but got 1 rows
> at org.apache.cassandra.cql3.CQLTester.assertEmpty(CQLTester.java:1089)
> at
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns(DeleteTest.java:463)
> at
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns(DeleteTest.java:427)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
