[jira] [Commented] (CASSANDRA-4206) AssertionError: originally calculated column size of 629444349 but now it is 588008950
[
https://issues.apache.org/jira/browse/CASSANDRA-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855152#comment-13855152
]
Christian Spriegel commented on CASSANDRA-4206:
---
I can confirm that this error also occurs with multithreaded_compaction=false.
AssertionError: originally calculated column size of 629444349 but now it is
588008950
--
Key: CASSANDRA-4206
URL: https://issues.apache.org/jira/browse/CASSANDRA-4206
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.9
Environment: Debian Squeeze Linux, kernel 2.6.32, sun-java6-bin
6.26-0squeeze1
Reporter: Patrik Modesto
I've 4 node cluster of Cassandra 1.0.9. There is a rfTest3 keyspace with RF=3
and one CF with two secondary indexes. I'm importing data into this CF using
Hadoop Mapreduce job, each row has less than 10 colkumns. From JMX:
MaxRowSize: 1597
MeanRowSize: 369
And there are some tens of millions of rows.
It's write-heavy usage and there is a big pressure on each node, there are
quite some dropped mutations on each node. After ~12 hours of inserting I see
these assertion exceptiona on 3 out of four nodes:
{noformat}
ERROR 06:25:40,124 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of 629444349 but
now it is 588008950
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:388)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:256)
at
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:84)
at
org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:437)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of
629444349 but now it is 588008950
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:384)
... 7 more
Caused by: java.lang.AssertionError: originally calculated column size
of 629444349 but now it is 588008950
at
org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at
org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:161)
at
org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:380)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{noformat}
Few lines regarding Hints from the output.log:
{noformat}
INFO 06:21:26,202 Compacting large row
system/HintsColumnFamily:7000 (1712834057 bytes)
incrementally
INFO 06:22:52,610 Compacting large row
system/HintsColumnFamily:1000 (2616073981 bytes)
incrementally
INFO 06:22:59,111 flushing high-traffic column family CFS(Keyspace='system',
ColumnFamily='HintsColumnFamily') (estimated 305147360 bytes)
INFO 06:22:59,813 Enqueuing flush of
Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes,
7452 ops)
INFO 06:22:59,814 Writing
Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes,
7452 ops)
{noformat}
I think the problem may be somehow connected to an IntegerType secondary
index. I had a different problem with CF with two secondary indexes, the
first UTF8Type, the second IntegerType. After a few hours of inserting data
in the afternoon and midnight repair+compact, the next day I couldn't find
any row using the IntegerType secondary index. The output was like this:
{noformat}
[default@rfTest3] get IndexTest where col1 =
'3230727:http://zaskolak.cz/download.php';
---
RowKey: 3230727:8383582:http://zaskolak.cz/download.php
= (column=col1, value=3230727:http://zaskolak.cz/download.php,
timestamp=1335348630332000)
= (column=col2, value=8383582, timestamp=1335348630332000)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855198#comment-13855198 ] Marcus Eriksson commented on CASSANDRA-5159: did you wait gc_grace time after doing the deletes? tombstones are not droppable until after that time. Expose droppable tombstone ratio stats over JMX --- Key: CASSANDRA-5159 URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.2.0 Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 1.2.1 Attachments: 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch Could be nice to be able to jmx-monitor the tombstone histogram collected during compactions to see how many gc-able tombstones there are left due to the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
Aleksey Yeschenko created CASSANDRA-6521: Summary: Thrift should validate SliceRange start and finish lengths Key: CASSANDRA-6521 URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 Project: Cassandra Issue Type: Bug Reporter: Ben Bromhead Assignee: Aleksey Yeschenko Fix For: 1.2.14, 2.0.4 To quote [~benbromhead]: bq. It appears that Cassandra does not check the length of a column name that is part of a range predicate for a *_slice query before it serialises the slice query to pass to the replicas. Names with a length greater than 0x cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. This further causes subsequent reads on the node to fail until Cassandra is restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
[ https://issues.apache.org/jira/browse/CASSANDRA-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6521: - Assignee: Ben Bromhead (was: Aleksey Yeschenko) Thrift should validate SliceRange start and finish lengths -- Key: CASSANDRA-6521 URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 Project: Cassandra Issue Type: Bug Reporter: Ben Bromhead Assignee: Ben Bromhead Fix For: 1.2.14, 2.0.4 To quote [~benbromhead]: bq. It appears that Cassandra does not check the length of a column name that is part of a range predicate for a *_slice query before it serialises the slice query to pass to the replicas. Names with a length greater than 0x cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. This further causes subsequent reads on the node to fail until Cassandra is restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
[ https://issues.apache.org/jira/browse/CASSANDRA-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6521: - Attachment: 6521.txt Attaching Ben's patch with several modifications. Thrift should validate SliceRange start and finish lengths -- Key: CASSANDRA-6521 URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 Project: Cassandra Issue Type: Bug Reporter: Ben Bromhead Assignee: Ben Bromhead Fix For: 1.2.14, 2.0.4 Attachments: 6521.txt To quote [~benbromhead]: bq. It appears that Cassandra does not check the length of a column name that is part of a range predicate for a *_slice query before it serialises the slice query to pass to the replicas. Names with a length greater than 0x cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. This further causes subsequent reads on the node to fail until Cassandra is restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
git commit: Validate SliceRange start and finish lengths
Updated Branches:
refs/heads/cassandra-1.2 924492345 - a58a95ddb
Validate SliceRange start and finish lengths
patch by Ben Bromhead; reviewed by Aleksey Yeschenko for CASSANDRA-6521
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58a95dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58a95dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58a95dd
Branch: refs/heads/cassandra-1.2
Commit: a58a95ddbb438cf8a2980d71b5cdbc2e63642685
Parents: 9244923
Author: Aleksey Yeschenko alek...@apache.org
Authored: Sun Dec 22 20:03:05 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Sun Dec 22 20:03:05 2013 +0300
--
CHANGES.txt | 1 +
.../org/apache/cassandra/thrift/ThriftValidation.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6f6c131..4eee4ee 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -4,6 +4,7 @@
* Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
* Don't drop local mutations without a trace (CASSANDRA-6510)
* Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
+ * Validate SliceRange start and finish lengths (CASSANDRA-6521)
1.2.13
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/src/java/org/apache/cassandra/thrift/ThriftValidation.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
index 5882e27..442ea4e 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
@@ -228,6 +228,15 @@ public class ThriftValidation
public static void validateRange(CFMetaData metadata, ColumnParent
column_parent, SliceRange range) throws
org.apache.cassandra.exceptions.InvalidRequestException
{
+if (range.count 0)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires
non-negative count);
+
+if (range.start.remaining() IColumn.MAX_NAME_LENGTH)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(range start length
cannot be larger than + IColumn.MAX_NAME_LENGTH);
+
+if (range.finish.remaining() IColumn.MAX_NAME_LENGTH)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(range finish length
cannot be larger than + IColumn.MAX_NAME_LENGTH);
+
AbstractType? comparator =
metadata.getComparatorFor(column_parent.super_column);
try
{
@@ -239,9 +248,6 @@ public class ThriftValidation
throw new
org.apache.cassandra.exceptions.InvalidRequestException(e.getMessage());
}
-if (range.count 0)
-throw new
org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires
non-negative count);
-
ComparatorByteBuffer orderedComparator = range.isReversed() ?
comparator.reverseComparator : comparator;
if (range.start.remaining() 0
range.finish.remaining() 0
[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/thrift/ThriftValidation.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03f4c92d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03f4c92d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03f4c92d
Branch: refs/heads/cassandra-2.0
Commit: 03f4c92d14ec7bbdac7524f531423c79015c8061
Parents: b5c9b49 a58a95d
Author: Aleksey Yeschenko alek...@apache.org
Authored: Sun Dec 22 20:09:18 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Sun Dec 22 20:09:18 2013 +0300
--
CHANGES.txt | 1 +
.../org/apache/cassandra/thrift/ThriftValidation.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/CHANGES.txt
--
diff --cc CHANGES.txt
index b80c821,4eee4ee..f38b58f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -34,50 -28,9 +34,51 @@@ Merged from 1.2
(CASSANDRA-6413)
* (Hadoop) add describe_local_ring (CASSANDRA-6268)
* Fix handling of concurrent directory creation failure (CASSANDRA-6459)
+ * Allow executing CREATE statements multiple times (CASSANDRA-6471)
+ * Don't send confusing info with timeouts (CASSANDRA-6491)
+ * Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
+ * Don't drop local mutations without a trace (CASSANDRA-6510)
+ * Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
++ * Validate SliceRange start and finish lengths (CASSANDRA-6521)
-1.2.12
+2.0.3
+ * Fix FD leak on slice read path (CASSANDRA-6275)
+ * Cancel read meter task when closing SSTR (CASSANDRA-6358)
+ * free off-heap IndexSummary during bulk (CASSANDRA-6359)
+ * Recover from IOException in accept() thread (CASSANDRA-6349)
+ * Improve Gossip tolerance of abnormally slow tasks (CASSANDRA-6338)
+ * Fix trying to hint timed out counter writes (CASSANDRA-6322)
+ * Allow restoring specific columnfamilies from archived CL (CASSANDRA-4809)
+ * Avoid flushing compaction_history after each operation (CASSANDRA-6287)
+ * Fix repair assertion error when tombstones expire (CASSANDRA-6277)
+ * Skip loading corrupt key cache (CASSANDRA-6260)
+ * Fixes for compacting larger-than-memory rows (CASSANDRA-6274)
+ * Compact hottest sstables first and optionally omit coldest from
+ compaction entirely (CASSANDRA-6109)
+ * Fix modifying column_metadata from thrift (CASSANDRA-6182)
+ * cqlsh: fix LIST USERS output (CASSANDRA-6242)
+ * Add IRequestSink interface (CASSANDRA-6248)
+ * Update memtable size while flushing (CASSANDRA-6249)
+ * Provide hooks around CQL2/CQL3 statement execution (CASSANDRA-6252)
+ * Require Permission.SELECT for CAS updates (CASSANDRA-6247)
+ * New CQL-aware SSTableWriter (CASSANDRA-5894)
+ * Reject CAS operation when the protocol v1 is used (CASSANDRA-6270)
+ * Correctly throw error when frame too large (CASSANDRA-5981)
+ * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299)
+ * Fix CQL3 table validation in Thrift (CASSANDRA-6140)
+ * Fix bug missing results with IN clauses (CASSANDRA-6327)
+ * Fix paging with reversed slices (CASSANDRA-6343)
+ * Set minTimestamp correctly to be able to drop expired sstables
(CASSANDRA-6337)
+ * Support NaN and Infinity as float literals (CASSANDRA-6003)
+ * Remove RF from nodetool ring output (CASSANDRA-6289)
+ * Fix attempting to flush empty rows (CASSANDRA-6374)
+ * Fix potential out of bounds exception when paging (CASSANDRA-6333)
+Merged from 1.2:
+ * Optimize FD phi calculation (CASSANDRA-6386)
+ * Improve initial FD phi estimate when starting up (CASSANDRA-6385)
+ * Don't list CQL3 table in CLI describe even if named explicitely
+ (CASSANDRA-5750)
* Invalidate row cache when dropping CF (CASSANDRA-6351)
* add non-jamm path for cached statements (CASSANDRA-6293)
* (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/src/java/org/apache/cassandra/thrift/ThriftValidation.java
--
diff --cc src/java/org/apache/cassandra/thrift/ThriftValidation.java
index 3a831f4,442ea4e..f2efc03
--- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
@@@ -268,7 -228,16 +268,16 @@@ public class ThriftValidatio
public static void validateRange(CFMetaData metadata, ColumnParent
column_parent, SliceRange range) throws
org.apache.cassandra.exceptions.InvalidRequestException
{
+ if (range.count 0)
+
[1/2] git commit: Validate SliceRange start and finish lengths
Updated Branches:
refs/heads/cassandra-2.0 b5c9b498f - 03f4c92d1
Validate SliceRange start and finish lengths
patch by Ben Bromhead; reviewed by Aleksey Yeschenko for CASSANDRA-6521
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58a95dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58a95dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58a95dd
Branch: refs/heads/cassandra-2.0
Commit: a58a95ddbb438cf8a2980d71b5cdbc2e63642685
Parents: 9244923
Author: Aleksey Yeschenko alek...@apache.org
Authored: Sun Dec 22 20:03:05 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Sun Dec 22 20:03:05 2013 +0300
--
CHANGES.txt | 1 +
.../org/apache/cassandra/thrift/ThriftValidation.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6f6c131..4eee4ee 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -4,6 +4,7 @@
* Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
* Don't drop local mutations without a trace (CASSANDRA-6510)
* Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
+ * Validate SliceRange start and finish lengths (CASSANDRA-6521)
1.2.13
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/src/java/org/apache/cassandra/thrift/ThriftValidation.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
index 5882e27..442ea4e 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
@@ -228,6 +228,15 @@ public class ThriftValidation
public static void validateRange(CFMetaData metadata, ColumnParent
column_parent, SliceRange range) throws
org.apache.cassandra.exceptions.InvalidRequestException
{
+if (range.count 0)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires
non-negative count);
+
+if (range.start.remaining() IColumn.MAX_NAME_LENGTH)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(range start length
cannot be larger than + IColumn.MAX_NAME_LENGTH);
+
+if (range.finish.remaining() IColumn.MAX_NAME_LENGTH)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(range finish length
cannot be larger than + IColumn.MAX_NAME_LENGTH);
+
AbstractType? comparator =
metadata.getComparatorFor(column_parent.super_column);
try
{
@@ -239,9 +248,6 @@ public class ThriftValidation
throw new
org.apache.cassandra.exceptions.InvalidRequestException(e.getMessage());
}
-if (range.count 0)
-throw new
org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires
non-negative count);
-
ComparatorByteBuffer orderedComparator = range.isReversed() ?
comparator.reverseComparator : comparator;
if (range.start.remaining() 0
range.finish.remaining() 0
[2/4] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/thrift/ThriftValidation.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03f4c92d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03f4c92d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03f4c92d
Branch: refs/heads/trunk
Commit: 03f4c92d14ec7bbdac7524f531423c79015c8061
Parents: b5c9b49 a58a95d
Author: Aleksey Yeschenko alek...@apache.org
Authored: Sun Dec 22 20:09:18 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Sun Dec 22 20:09:18 2013 +0300
--
CHANGES.txt | 1 +
.../org/apache/cassandra/thrift/ThriftValidation.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/CHANGES.txt
--
diff --cc CHANGES.txt
index b80c821,4eee4ee..f38b58f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -34,50 -28,9 +34,51 @@@ Merged from 1.2
(CASSANDRA-6413)
* (Hadoop) add describe_local_ring (CASSANDRA-6268)
* Fix handling of concurrent directory creation failure (CASSANDRA-6459)
+ * Allow executing CREATE statements multiple times (CASSANDRA-6471)
+ * Don't send confusing info with timeouts (CASSANDRA-6491)
+ * Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
+ * Don't drop local mutations without a trace (CASSANDRA-6510)
+ * Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
++ * Validate SliceRange start and finish lengths (CASSANDRA-6521)
-1.2.12
+2.0.3
+ * Fix FD leak on slice read path (CASSANDRA-6275)
+ * Cancel read meter task when closing SSTR (CASSANDRA-6358)
+ * free off-heap IndexSummary during bulk (CASSANDRA-6359)
+ * Recover from IOException in accept() thread (CASSANDRA-6349)
+ * Improve Gossip tolerance of abnormally slow tasks (CASSANDRA-6338)
+ * Fix trying to hint timed out counter writes (CASSANDRA-6322)
+ * Allow restoring specific columnfamilies from archived CL (CASSANDRA-4809)
+ * Avoid flushing compaction_history after each operation (CASSANDRA-6287)
+ * Fix repair assertion error when tombstones expire (CASSANDRA-6277)
+ * Skip loading corrupt key cache (CASSANDRA-6260)
+ * Fixes for compacting larger-than-memory rows (CASSANDRA-6274)
+ * Compact hottest sstables first and optionally omit coldest from
+ compaction entirely (CASSANDRA-6109)
+ * Fix modifying column_metadata from thrift (CASSANDRA-6182)
+ * cqlsh: fix LIST USERS output (CASSANDRA-6242)
+ * Add IRequestSink interface (CASSANDRA-6248)
+ * Update memtable size while flushing (CASSANDRA-6249)
+ * Provide hooks around CQL2/CQL3 statement execution (CASSANDRA-6252)
+ * Require Permission.SELECT for CAS updates (CASSANDRA-6247)
+ * New CQL-aware SSTableWriter (CASSANDRA-5894)
+ * Reject CAS operation when the protocol v1 is used (CASSANDRA-6270)
+ * Correctly throw error when frame too large (CASSANDRA-5981)
+ * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299)
+ * Fix CQL3 table validation in Thrift (CASSANDRA-6140)
+ * Fix bug missing results with IN clauses (CASSANDRA-6327)
+ * Fix paging with reversed slices (CASSANDRA-6343)
+ * Set minTimestamp correctly to be able to drop expired sstables
(CASSANDRA-6337)
+ * Support NaN and Infinity as float literals (CASSANDRA-6003)
+ * Remove RF from nodetool ring output (CASSANDRA-6289)
+ * Fix attempting to flush empty rows (CASSANDRA-6374)
+ * Fix potential out of bounds exception when paging (CASSANDRA-6333)
+Merged from 1.2:
+ * Optimize FD phi calculation (CASSANDRA-6386)
+ * Improve initial FD phi estimate when starting up (CASSANDRA-6385)
+ * Don't list CQL3 table in CLI describe even if named explicitely
+ (CASSANDRA-5750)
* Invalidate row cache when dropping CF (CASSANDRA-6351)
* add non-jamm path for cached statements (CASSANDRA-6293)
* (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/src/java/org/apache/cassandra/thrift/ThriftValidation.java
--
diff --cc src/java/org/apache/cassandra/thrift/ThriftValidation.java
index 3a831f4,442ea4e..f2efc03
--- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
@@@ -268,7 -228,16 +268,16 @@@ public class ThriftValidatio
public static void validateRange(CFMetaData metadata, ColumnParent
column_parent, SliceRange range) throws
org.apache.cassandra.exceptions.InvalidRequestException
{
+ if (range.count 0)
+
[4/4] git commit: Fix merge
Fix merge Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/34235ad7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/34235ad7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/34235ad7 Branch: refs/heads/trunk Commit: 34235ad7b5722a5a5ee9680ba3a2d563a67ecc6f Parents: 85863e2 Author: Aleksey Yeschenko alek...@apache.org Authored: Sun Dec 22 20:10:59 2013 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sun Dec 22 20:10:59 2013 +0300 -- src/java/org/apache/cassandra/thrift/ThriftValidation.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/34235ad7/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java b/src/java/org/apache/cassandra/thrift/ThriftValidation.java index b597203..d491636 100644 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@ -268,7 +268,7 @@ public class ThriftValidation if (range.count 0) throw new org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires non-negative count); -int maxNameLength = org.apache.cassandra.db.Column.MAX_NAME_LENGTH; +int maxNameLength = Cell.MAX_NAME_LENGTH; if (range.start.remaining() maxNameLength) throw new org.apache.cassandra.exceptions.InvalidRequestException(range start length cannot be larger than + maxNameLength); if (range.finish.remaining() maxNameLength)
[3/4] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85863e27 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85863e27 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85863e27 Branch: refs/heads/trunk Commit: 85863e273ecc5808fc9085b0f8f63aa02c4be456 Parents: d0ab751 03f4c92 Author: Aleksey Yeschenko alek...@apache.org Authored: Sun Dec 22 20:09:57 2013 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Sun Dec 22 20:09:57 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/85863e27/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/85863e27/src/java/org/apache/cassandra/thrift/ThriftValidation.java --
[1/4] git commit: Validate SliceRange start and finish lengths
Updated Branches:
refs/heads/trunk d0ab7519c - 34235ad7b
Validate SliceRange start and finish lengths
patch by Ben Bromhead; reviewed by Aleksey Yeschenko for CASSANDRA-6521
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58a95dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58a95dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58a95dd
Branch: refs/heads/trunk
Commit: a58a95ddbb438cf8a2980d71b5cdbc2e63642685
Parents: 9244923
Author: Aleksey Yeschenko alek...@apache.org
Authored: Sun Dec 22 20:03:05 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Sun Dec 22 20:03:05 2013 +0300
--
CHANGES.txt | 1 +
.../org/apache/cassandra/thrift/ThriftValidation.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6f6c131..4eee4ee 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -4,6 +4,7 @@
* Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
* Don't drop local mutations without a trace (CASSANDRA-6510)
* Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
+ * Validate SliceRange start and finish lengths (CASSANDRA-6521)
1.2.13
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/src/java/org/apache/cassandra/thrift/ThriftValidation.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
index 5882e27..442ea4e 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
@@ -228,6 +228,15 @@ public class ThriftValidation
public static void validateRange(CFMetaData metadata, ColumnParent
column_parent, SliceRange range) throws
org.apache.cassandra.exceptions.InvalidRequestException
{
+if (range.count 0)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires
non-negative count);
+
+if (range.start.remaining() IColumn.MAX_NAME_LENGTH)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(range start length
cannot be larger than + IColumn.MAX_NAME_LENGTH);
+
+if (range.finish.remaining() IColumn.MAX_NAME_LENGTH)
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(range finish length
cannot be larger than + IColumn.MAX_NAME_LENGTH);
+
AbstractType? comparator =
metadata.getComparatorFor(column_parent.super_column);
try
{
@@ -239,9 +248,6 @@ public class ThriftValidation
throw new
org.apache.cassandra.exceptions.InvalidRequestException(e.getMessage());
}
-if (range.count 0)
-throw new
org.apache.cassandra.exceptions.InvalidRequestException(get_slice requires
non-negative count);
-
ComparatorByteBuffer orderedComparator = range.isReversed() ?
comparator.reverseComparator : comparator;
if (range.start.remaining() 0
range.finish.remaining() 0
[jira] [Resolved] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
[ https://issues.apache.org/jira/browse/CASSANDRA-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-6521. -- Resolution: Fixed Committed, thanks, Ben. Thrift should validate SliceRange start and finish lengths -- Key: CASSANDRA-6521 URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 Project: Cassandra Issue Type: Bug Reporter: Ben Bromhead Assignee: Ben Bromhead Fix For: 1.2.14, 2.0.4 Attachments: 6521.txt To quote [~benbromhead]: bq. It appears that Cassandra does not check the length of a column name that is part of a range predicate for a *_slice query before it serialises the slice query to pass to the replicas. Names with a length greater than 0x cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. This further causes subsequent reads on the node to fail until Cassandra is restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855223#comment-13855223 ] Daniel Kador commented on CASSANDRA-5159: - Yes, gc_grace is currently set to 1 day and we've waited that long. Expose droppable tombstone ratio stats over JMX --- Key: CASSANDRA-5159 URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.2.0 Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 1.2.1 Attachments: 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch Could be nice to be able to jmx-monitor the tombstone histogram collected during compactions to see how many gc-able tombstones there are left due to the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4206) AssertionError: originally calculated column size of 629444349 but now it is 588008950
[
https://issues.apache.org/jira/browse/CASSANDRA-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855228#comment-13855228
]
Daniel Kador commented on CASSANDRA-4206:
-
We got desperate and tried multithreaded_compaction=false yesterday afternoon
and it seems to have worked for us. Pending compactions across the cluster
have dropped from over 20,000 to under 600 (we're not quite done yet). The
numbers were so high because we added six new nodes and they were unable to
compact quickly until we made this settings change.
No idea why it worked for us but thought I'd share our experience. If there's
any other useful information I can share, let me know.
AssertionError: originally calculated column size of 629444349 but now it is
588008950
--
Key: CASSANDRA-4206
URL: https://issues.apache.org/jira/browse/CASSANDRA-4206
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.9
Environment: Debian Squeeze Linux, kernel 2.6.32, sun-java6-bin
6.26-0squeeze1
Reporter: Patrik Modesto
I've 4 node cluster of Cassandra 1.0.9. There is a rfTest3 keyspace with RF=3
and one CF with two secondary indexes. I'm importing data into this CF using
Hadoop Mapreduce job, each row has less than 10 colkumns. From JMX:
MaxRowSize: 1597
MeanRowSize: 369
And there are some tens of millions of rows.
It's write-heavy usage and there is a big pressure on each node, there are
quite some dropped mutations on each node. After ~12 hours of inserting I see
these assertion exceptiona on 3 out of four nodes:
{noformat}
ERROR 06:25:40,124 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of 629444349 but
now it is 588008950
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:388)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:256)
at
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:84)
at
org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:437)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of
629444349 but now it is 588008950
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:384)
... 7 more
Caused by: java.lang.AssertionError: originally calculated column size
of 629444349 but now it is 588008950
at
org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at
org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:161)
at
org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:380)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{noformat}
Few lines regarding Hints from the output.log:
{noformat}
INFO 06:21:26,202 Compacting large row
system/HintsColumnFamily:7000 (1712834057 bytes)
incrementally
INFO 06:22:52,610 Compacting large row
system/HintsColumnFamily:1000 (2616073981 bytes)
incrementally
INFO 06:22:59,111 flushing high-traffic column family CFS(Keyspace='system',
ColumnFamily='HintsColumnFamily') (estimated 305147360 bytes)
INFO 06:22:59,813 Enqueuing flush of
Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes,
7452 ops)
INFO 06:22:59,814 Writing
Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes,
7452 ops)
{noformat}
I think the problem may be somehow connected to an IntegerType secondary
index. I had a different problem with CF with two secondary indexes, the
first UTF8Type, the second IntegerType. After a few hours of inserting data
in the afternoon and midnight repair+compact, the next day I couldn't find
any row
[jira] [Commented] (CASSANDRA-6236) Update native protocol server to Netty 4
[ https://issues.apache.org/jira/browse/CASSANDRA-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855229#comment-13855229 ] Aleksey Yeschenko commented on CASSANDRA-6236: -- Or maybe skip Netty 4 and go directly to Netty 5 - http://netty.io/wiki/new-and-noteworthy-in-5.x.html Update native protocol server to Netty 4 Key: CASSANDRA-6236 URL: https://issues.apache.org/jira/browse/CASSANDRA-6236 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Priority: Minor Fix For: 2.1 We should switch to Netty 4 at some point, since it's the future. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855247#comment-13855247 ] Marcus Eriksson commented on CASSANDRA-5159: i have checked this attribute on clusters and it has worked then (getting reasonable values), versions probably a bit later than yours (1.2.11) if you could describe your config etc in a new ticket i could look in to it a bit closer. Expose droppable tombstone ratio stats over JMX --- Key: CASSANDRA-5159 URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.2.0 Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 1.2.1 Attachments: 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch Could be nice to be able to jmx-monitor the tombstone histogram collected during compactions to see how many gc-able tombstones there are left due to the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855248#comment-13855248 ] Daniel Kador commented on CASSANDRA-5159: - Filed CASSANDRA-6522. Thanks! Expose droppable tombstone ratio stats over JMX --- Key: CASSANDRA-5159 URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 Project: Cassandra Issue Type: Improvement Components: Tools Affects Versions: 1.2.0 Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 1.2.1 Attachments: 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch Could be nice to be able to jmx-monitor the tombstone histogram collected during compactions to see how many gc-able tombstones there are left due to the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6522) DroppableTombstoneRatio JMX value is 0.0 for all CFs
Daniel Kador created CASSANDRA-6522: --- Summary: DroppableTombstoneRatio JMX value is 0.0 for all CFs Key: CASSANDRA-6522 URL: https://issues.apache.org/jira/browse/CASSANDRA-6522 Project: Cassandra Issue Type: Bug Components: Core Environment: Ubuntu 12.04 LTS, Cassandra 1.2.8 Reporter: Daniel Kador Priority: Minor We're seeing that the JMX value for DroppableTombstoneRatio for all our CFs is 0.0. On the face of it that seems wrong since we've definitely issued a ton of deletes for row keys to expire some old data that we no longer need (and it definitely hasn't been reclaimed from disk yet). Am I misunderstanding what this means / how to use it? We're on 1.2.8 and using leveled compaction for all our CFs. gc_grace_seconds is set to 1 day and we've issued a series of deletes over a day ago, so gc_grace has elapsed. Cluster is 18 nodes. Two DCs, so 9 nodes in each DC. Each node has capacity for 1.5TB or so and is sitting with about 1TB under management. That's why we wanted to do deletes, obviously. Most of that 1TB is a single CF (called events) which represents intermediate state for us that we can delete. Happy to provide any more info, just let me know. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6450) sstable2json hangs if keyspace uses authentication
[ https://issues.apache.org/jira/browse/CASSANDRA-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855447#comment-13855447 ] Daniel Kador commented on CASSANDRA-6450: - Just wanted to update this to say that this is blocking us from doing some investigation into whether or not we're getting hit by a tombstone. Would love to hear if this shouldn't work and we're using the tool wrong or if it should and it's a bug. Thanks! sstable2json hangs if keyspace uses authentication -- Key: CASSANDRA-6450 URL: https://issues.apache.org/jira/browse/CASSANDRA-6450 Project: Cassandra Issue Type: Bug Environment: Ubuntu 12 Reporter: Josh Dzielak Priority: Minor Running sstable2json against an authenticated keyspace hangs indefinitely. True for other utilities based on SSTableExport as well. Running sstable2json against other unauthenticated keyspaces in the same node/cluster was successful. Running against any CF in the keyspace with password authentication on resulted in a hang. It looks like it gets about to: Table table = Table.open(descriptor.ksname); or table.getColumnFamilyStore(baseName); in SSTableExport.java but no farther. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
