[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-8603: -- Component/s: Local Write-Read Paths > Cut tombstone memory footprint in half for cql deletes > -- > > Key: CASSANDRA-8603 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Dominic Letz >Assignee: Benjamin Lerer > Labels: tombstone > Fix For: 2.1.6, 2.2.0 rc1 > > Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, > cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log > > > As CQL does not yet support range deletes every delete from CQL results in a > "Semi-RangeTombstone" which actually has the same start and end values - but > until today they are copies. Effectively doubling the required heap memory to > store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-8603: -- Attachment: 8603-2.1-V3.txt Patch for 2.1 that perform the check during deserialization Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Dominic Letz Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8603: - Assignee: Benjamin Lerer (was: Dominic Letz) Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Benjamin Lerer Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8603: - Reviewer: Aleksey Yeschenko (was: Benjamin Lerer) Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Assignee: Benjamin Lerer Labels: tombstone Attachments: 8603-2.1-V3.txt, cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-8603: -- Reviewer: Benjamin Lerer Reproduced In: 2.1.2, 2.0.11 (was: 2.0.11, 2.1.2) Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Labels: tombstone Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, cassandra-2.1-8603_v2.txt, system.log As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[
https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dominic Letz updated CASSANDRA-8603:
Attachment: cassandra-2.1-8603_v2.txt
The equality in the system.log was in the system keyspace
(system/local-7ad54392bcdd35a684174e047860b377)
This is because of the code in RangeTombstone.java:242
{code}
RangeTombstone t = new RangeTombstone(cell.name(), cell.name(),
cell.timestamp(), 0);
{code}
I've provided a patch v2 that actually catches the case I intended and replaces
the duplicated ByteBuffer by using the Composite.end() method to construct a
BoundedComposite referencing the original.
{code}
super(start, (start != stop stop.equals(start.end())) ? start.end() : stop,
delTime);
{code}
Cut tombstone memory footprint in half for cql deletes
--
Key: CASSANDRA-8603
URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Dominic Letz
Labels: tombstone
Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt,
cassandra-2.1-8603_v2.txt, system.log
As CQL does not yet support range deletes every delete from CQL results in a
Semi-RangeTombstone which actually has the same start and end values - but
until today they are copies. Effectively doubling the required heap memory to
store the RangeTombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[
https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dominic Letz updated CASSANDRA-8603:
Attachment: system.log
Sorry for the long delay here. I did testing before but your comment made me
look deeper and indeed something is wrong.
First off the attached patches do not catch the situation that they were
intended to catch - but they do catch something and that mislead me. So maybe
there is a bug as you pointed out.
The test case I'm using is like this:
{code}
CREATE KEYSPACE test
WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 };
CREATE TABLE test.timeseries (name text, timestamp int, value text, PRIMARY KEY
((name), timestamp))
WITH compaction = {'class': 'SizeTieredCompactionStrategy'} AND CLUSTERING
ORDER BY (timestamp DESC);
INSERT INTO test.timeseries (name, timestamp, value) VALUES('a', 1, 'hello');
INSERT INTO test.timeseries (name, timestamp, value) VALUES('a', 2, 'world');
INSERT INTO test.timeseries (name, timestamp, value) VALUES('a', 5, 'hello
world');
DELETE FROM test.timeseries WHERE name = 'a' AND timestamp = 1;
DELETE FROM test.timeseries WHERE name = 'a' AND timestamp = 2;
DELETE FROM test.timeseries WHERE name = 'a' AND timestamp = 5;
{code}
Plus adding this log line next to the initial patch content:
{code}
public RangeTombstone(Composite start, Composite stop, DeletionTime delTime)
{
super(start, stop.equals(start) ? start : stop, delTime);
LoggerFactory.getLogger(RangeTombstone.class).error(String.format(Test:
start.equals(stop) %s (%s == %s) %s, , start.equals(stop), new
String(Hex.encodeHex(start.toByteBuffer().array())), new
String(Hex.encodeHex(stop.toByteBuffer().array())),
start.toByteBuffer().compareTo(stop.toByteBuffer(;
}
{/code}
Then I wiped the cassandra data directory and started from scratch, and once
started ran the said cql. The resulting system.log is attached
It seems that just during start-up there are a couple of hits into the new
logic where the tombstone .start and .end are actually exactly the same as
shown by the debug output:
{code}
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,476 RangeTombstone.java:48 -
Test: start.equals(stop) true (00 == 00) 0,
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,478 RangeTombstone.java:48 -
Test: start.equals(stop) false (0006746f6b656e73ff == 0006746f6b656e7301) -2,
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,480 RangeTombstone.java:48 -
Test: start.equals(stop) true (000c626f6f74737472617070656400 ==
000c626f6f74737472617070656400) 0,
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,481 RangeTombstone.java:48 -
Test: start.equals(stop) true (000c636c75737465725f6e616d6500 ==
000c636c75737465725f6e616d6500) 0,
...
{code}
But then further down in the log when the actual test cql code is executed
these messages pop up:
{code}
ERROR [Thrift:1] 2015-01-16 08:51:14,089 RangeTombstone.java:48 - Test:
start.equals(stop) false (00040001ff == 0004000101) -2,
ERROR [Thrift:1] 2015-01-16 08:51:14,092 RangeTombstone.java:48 - Test:
start.equals(stop) false (00040002ff == 0004000201) -2,
ERROR [Thrift:1] 2015-01-16 08:51:14,094 RangeTombstone.java:48 - Test:
start.equals(stop) false (00040005ff == 0004000501) -2,
{code}
Show that even though they keys match up until the provided sample value (1, 2,
5) there is an additional byte that is creating a difference and thus the code
is not catching. It seems I wasn't deep enough into the code to actually make
my proposed change successfully as I don't know the meaning of that last byte -
but the concept might still hold?
Cut tombstone memory footprint in half for cql deletes
--
Key: CASSANDRA-8603
URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Dominic Letz
Labels: tombstone
Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt,
system.log
As CQL does not yet support range deletes every delete from CQL results in a
Semi-RangeTombstone which actually has the same start and end values - but
until today they are copies. Effectively doubling the required heap memory to
store the RangeTombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Letz updated CASSANDRA-8603: Attachment: cassandra-2.0.11-8603.txt Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Labels: tombstone Attachments: cassandra-2.0.11-8603.txt As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
[ https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Letz updated CASSANDRA-8603: Attachment: cassandra-2.1-8603.txt Attached trivial patches for 2.0.11 and 2.1 Cut tombstone memory footprint in half for cql deletes -- Key: CASSANDRA-8603 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dominic Letz Labels: tombstone Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt As CQL does not yet support range deletes every delete from CQL results in a Semi-RangeTombstone which actually has the same start and end values - but until today they are copies. Effectively doubling the required heap memory to store the RangeTombstone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
