[jira] [Updated] (CASSANDRA-8660) Cassandra makes Java 1.7+ SEGFAULT
[ https://issues.apache.org/jira/browse/CASSANDRA-8660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Belanger updated CASSANDRA-8660: --- Attachment: hs_err_pid-patch1.log Here is the hs_err_pid file with the first proposed patch. Cassandra makes Java 1.7+ SEGFAULT -- Key: CASSANDRA-8660 URL: https://issues.apache.org/jira/browse/CASSANDRA-8660 Project: Cassandra Issue Type: Bug Components: Core Environment: Linux Redhat 6.5 Reporter: Pierre Belanger Priority: Critical Attachments: 8660, hs_err_pid-patch1.log, hs_err_pid12004.log We upgraded a 3 nodes cluster from Cassandra 2.0.11-5f54285e9e to 2.1-7e6d9eb842. With Cassandra 2.1 , within ~10 min. cassandra manages to makes Java 1.7.0_72-b14 segfault. This is also tested and same behavior w/ Java 1.7.0_60-b19 and 1.8.0_25-b17. We were able to use same data, downgrade back to 2.0 and things ran for ~3-4 days w/ no hick up. Upgrading back to Cassandra-2.1, Java does a segfault again. See attached hs_err file. Pierre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285916#comment-14285916 ] Ariel Weisberg commented on CASSANDRA-7438: --- If you turn the OOME throwing on in C* I am +1. I did a quick performance test with the cache and compared it to the SerializingCache. I didn't test a scenario where it would be better/faster, but the performance looked just as good. Very noisy test with different results every time I restarted so maybe not a great way to measure. Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Robert Stupp Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch, tests.zip Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286083#comment-14286083 ] Aleksey Yeschenko commented on CASSANDRA-8662: -- Would love to, but not sure if we can do it without breaking backward compatibility in a minor release (looking at pluggable metrics reporting breakage). Also, it's a reasonable amount of codes, probably too much for a 2.1 minor :( Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: CASSANDRA-8662v1.txt cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-8662: - Attachment: CASSANDRA-8662v1.txt Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: CASSANDRA-8662v1.txt cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
[
https://issues.apache.org/jira/browse/CASSANDRA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286130#comment-14286130
]
Andrew J Lenards commented on CASSANDRA-8663:
-
I should also note that doing the same operations in cqlsh for (DSE 4.6.0 /
Cassandra 2.0.11.83) does not yield the integer behavior.
{noformat}
cqlsh:system CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh:system CONSISTENCY;
Current consistency level is QUORUM.
cqlsh:system CONSISTENCY EACH_QUORUM;
Consistency level set to EACH_QUORUM.
{noformat}
cqlsh is showing the integer value for ConsistencyLevel not name
-
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
Labels: lhf
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The
name of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
[
https://issues.apache.org/jira/browse/CASSANDRA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8663:
Labels: lhf (was: )
cqlsh is showing the integer value for ConsistencyLevel not name
-
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
Labels: lhf
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The
name of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-7410: Attachment: 7410-2.1-branch.txt patch on cassandra-2.1 is attached. It can be used to debug the streaming error as shown above. Pig support for BulkOutputFormat as a parameter in url -- Key: CASSANDRA-7410 URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Alex Liu Assignee: Alex Liu Priority: Minor Fix For: 2.0.13 Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/732986bb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/732986bb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/732986bb Branch: refs/heads/trunk Commit: 732986bbd02a5acafaf97de24ea555c8d426b873 Parents: 0a09b87 0c2eaa9 Author: Benjamin Lerer b_le...@hotmail.com Authored: Wed Jan 21 20:19:42 2015 +0100 Committer: Robert Stupp sn...@snazy.de Committed: Wed Jan 21 20:19:42 2015 +0100 -- CHANGES.txt | 3 ++- .../cassandra/cql3/MultiColumnRelationTest.java | 15 + .../cql3/SingleColumnRelationTest.java | 22 ++-- 3 files changed, 37 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/732986bb/CHANGES.txt -- diff --cc CHANGES.txt index 9343d66,9cd8189..cc6478e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,58 -1,5 +1,59 @@@ -2.1.3 +3.0 + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Add role based access control (CASSANDRA-7653) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead -of IN value list order (CASSANDRA-7981) ++ of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * Improve concurrency of repair (CASSANDRA-6455, 8208) + + +2.1.3 * Add tooling to detect hot partitions (CASSANDRA-7974) * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) http://git-wip-us.apache.org/repos/asf/cassandra/blob/732986bb/test/unit/org/apache/cassandra/cql3/MultiColumnRelationTest.java -- diff --cc test/unit/org/apache/cassandra/cql3/MultiColumnRelationTest.java index
[1/3] cassandra git commit: Duplicate rows returned when in clause has repeated values
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 faf91818b - 0c2eaa9cb
refs/heads/trunk 0a09b87dc - 732986bbd
Duplicate rows returned when in clause has repeated values
Patch by Benjamin Lerer, reviewed by Robert Stupp for CASSANDRA-6706
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0c2eaa9c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0c2eaa9c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0c2eaa9c
Branch: refs/heads/cassandra-2.1
Commit: 0c2eaa9cbb51f064b439c4d098adb5aa76b65b0f
Parents: faf9181
Author: Benjamin Lerer b_le...@hotmail.com
Authored: Wed Jan 21 20:12:55 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Wed Jan 21 20:12:55 2015 +0100
--
CHANGES.txt | 1 +
.../cql3/statements/SelectStatement.java| 25
2 files changed, 26 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0c2eaa9c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f1eaa77..9cd8189 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707)
* Add tooling to detect hot partitions (CASSANDRA-7974)
* Fix cassandra-stress user-mode truncation of partition generation
(CASSANDRA-8608)
* Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0c2eaa9c/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 30259dd..633d43c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -65,6 +65,13 @@ public class SelectStatement implements CQLStatement
private static final int DEFAULT_COUNT_PAGE_SIZE = 1;
+/**
+ * In the current version a query containing duplicate values in an IN
restriction on the partition key will
+ * cause the same record to be returned multiple time. This behavior will
be changed in 3.0 but until then
+ * we will log a warning the first time this problem occurs.
+ */
+private static volatile boolean
HAS_LOGGED_WARNING_FOR_IN_RESTRICTION_WITH_DUPLICATES;
+
private final int boundTerms;
public final CFMetaData cfm;
public final Parameters parameters;
@@ -588,6 +595,13 @@ public class SelectStatement implements CQLStatement
if (builder.remainingCount() == 1)
{
+if (values.size() 1
!HAS_LOGGED_WARNING_FOR_IN_RESTRICTION_WITH_DUPLICATES
containsDuplicates(values))
+{
+// This approach does not fully prevent race conditions
but it is not a big deal.
+HAS_LOGGED_WARNING_FOR_IN_RESTRICTION_WITH_DUPLICATES =
true;
+logger.warn(SELECT queries with IN restrictions on the
partition key containing duplicate values will return duplicate rows.);
+}
+
for (ByteBuffer val : values)
{
if (val == null)
@@ -609,6 +623,17 @@ public class SelectStatement implements CQLStatement
return keys;
}
+/**
+ * Checks if the specified list contains duplicate values.
+ *
+ * @param values the values to check
+ * @return codetrue/code if the specified list contains duplicate
values, codefalse/code otherwise.
+ */
+private static boolean containsDuplicates(ListByteBuffer values)
+{
+return new HashSet(values).size() values.size();
+}
+
private ByteBuffer getKeyBound(Bound b, QueryOptions options) throws
InvalidRequestException
{
// Deal with unrestricted partition key components (special-casing is
required to deal with 2i queries on the first
[jira] [Commented] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286112#comment-14286112 ] Aleksey Yeschenko commented on CASSANDRA-8662: -- Oh well. My bad for not looking into it well enough (haven't noticed the patch, thought you wanted to backport CASSANDRA-5657. This - we can do. Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: CASSANDRA-8662v1.txt cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] cassandra git commit: Duplicate rows returned when in clause has repeated values
Duplicate rows returned when in clause has repeated values
Patch by Benjamin Lerer, reviewed by Robert Stupp for CASSANDRA-6706
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0c2eaa9c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0c2eaa9c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0c2eaa9c
Branch: refs/heads/trunk
Commit: 0c2eaa9cbb51f064b439c4d098adb5aa76b65b0f
Parents: faf9181
Author: Benjamin Lerer b_le...@hotmail.com
Authored: Wed Jan 21 20:12:55 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Wed Jan 21 20:12:55 2015 +0100
--
CHANGES.txt | 1 +
.../cql3/statements/SelectStatement.java| 25
2 files changed, 26 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0c2eaa9c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f1eaa77..9cd8189 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707)
* Add tooling to detect hot partitions (CASSANDRA-7974)
* Fix cassandra-stress user-mode truncation of partition generation
(CASSANDRA-8608)
* Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0c2eaa9c/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 30259dd..633d43c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -65,6 +65,13 @@ public class SelectStatement implements CQLStatement
private static final int DEFAULT_COUNT_PAGE_SIZE = 1;
+/**
+ * In the current version a query containing duplicate values in an IN
restriction on the partition key will
+ * cause the same record to be returned multiple time. This behavior will
be changed in 3.0 but until then
+ * we will log a warning the first time this problem occurs.
+ */
+private static volatile boolean
HAS_LOGGED_WARNING_FOR_IN_RESTRICTION_WITH_DUPLICATES;
+
private final int boundTerms;
public final CFMetaData cfm;
public final Parameters parameters;
@@ -588,6 +595,13 @@ public class SelectStatement implements CQLStatement
if (builder.remainingCount() == 1)
{
+if (values.size() 1
!HAS_LOGGED_WARNING_FOR_IN_RESTRICTION_WITH_DUPLICATES
containsDuplicates(values))
+{
+// This approach does not fully prevent race conditions
but it is not a big deal.
+HAS_LOGGED_WARNING_FOR_IN_RESTRICTION_WITH_DUPLICATES =
true;
+logger.warn(SELECT queries with IN restrictions on the
partition key containing duplicate values will return duplicate rows.);
+}
+
for (ByteBuffer val : values)
{
if (val == null)
@@ -609,6 +623,17 @@ public class SelectStatement implements CQLStatement
return keys;
}
+/**
+ * Checks if the specified list contains duplicate values.
+ *
+ * @param values the values to check
+ * @return codetrue/code if the specified list contains duplicate
values, codefalse/code otherwise.
+ */
+private static boolean containsDuplicates(ListByteBuffer values)
+{
+return new HashSet(values).size() values.size();
+}
+
private ByteBuffer getKeyBound(Bound b, QueryOptions options) throws
InvalidRequestException
{
// Deal with unrestricted partition key components (special-casing is
required to deal with 2i queries on the first
[jira] [Updated] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8662: - Reviewer: Aleksey Yeschenko Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: CASSANDRA-8662v1.txt cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
Andrew J Lenards created CASSANDRA-8663:
---
Summary: cqlsh is showing the integer value for ConsistencyLevel
not name
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
When you use issues the ``CONSISTENCY;`` command in `cqlsh`, it is showing the
integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The name
of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[2/3] cassandra git commit: Add tooling to detect hot partitions
Add tooling to detect hot partitions
Patch by Chris Lohfink, reviewed by brandonwilliams for CASSANDRA-7974
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/faf91818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/faf91818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/faf91818
Branch: refs/heads/trunk
Commit: faf91818b46fb51ed576664a1119315e7b7c3383
Parents: 1435b9a
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jan 21 11:45:45 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jan 21 11:45:45 2015 -0600
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 69 -
.../cassandra/db/ColumnFamilyStoreMBean.java| 14 ++
.../cassandra/metrics/ColumnFamilyMetrics.java | 19 ++-
.../org/apache/cassandra/tools/NodeProbe.java | 26 +++-
.../org/apache/cassandra/tools/NodeTool.java| 83 ++-
.../org/apache/cassandra/utils/TopKSampler.java | 139 ++
.../apache/cassandra/utils/TopKSamplerTest.java | 147 +++
8 files changed, 482 insertions(+), 16 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/faf91818/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0c2bab8..f1eaa77 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Add tooling to detect hot partitions (CASSANDRA-7974)
* Fix cassandra-stress user-mode truncation of partition generation
(CASSANDRA-8608)
* Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
* Don't allow starting multiple inc repairs on the same sstables
(CASSANDRA-8316)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/faf91818/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index f7a691e..0c95b0e 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -26,14 +26,12 @@ import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;
import java.util.regex.Pattern;
import javax.management.*;
+import javax.management.openmbean.*;
import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Predicate;
+import com.google.common.base.*;
import com.google.common.collect.*;
import com.google.common.util.concurrent.*;
-import com.google.common.util.concurrent.Futures;
-import com.google.common.util.concurrent.Uninterruptibles;
import org.json.simple.*;
import org.slf4j.Logger;
@@ -68,14 +66,18 @@ import
org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataType;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.metrics.ColumnFamilyMetrics;
+import org.apache.cassandra.metrics.ColumnFamilyMetrics.Sampler;
import org.apache.cassandra.service.CacheService;
import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.streaming.StreamLockfile;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.TopKSampler.SamplerResult;
import org.apache.cassandra.utils.concurrent.OpOrder;
import org.apache.cassandra.utils.memory.MemtableAllocator;
+import com.clearspring.analytics.stream.Counter;
+
public class ColumnFamilyStore implements ColumnFamilyStoreMBean
{
private static final Logger logger =
LoggerFactory.getLogger(ColumnFamilyStore.class);
@@ -102,6 +104,39 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
new NamedThreadFactory(MemtableReclaimMemory),
internal);
+private static final String[] COUNTER_NAMES = new String[]{raw, count,
error, string};
+private static final String[] COUNTER_DESCS = new String[]
+{ partition key in raw hex bytes,
+ value of this partition for given sampler,
+ value is within the error bounds plus or minus of this,
+ the partition key turned into a human readable format };
+private static final CompositeType COUNTER_COMPOSITE_TYPE;
+private static final TabularType COUNTER_TYPE;
+
+private static final String[] SAMPLER_NAMES = new String[]{cardinality,
partitions};
+private static final String[]
[jira] [Commented] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286105#comment-14286105 ] Chris Lohfink commented on CASSANDRA-8662: -- In attached patch the output would be the same, just changing from backing it with the Histogram to using the EstimatedHistogram (which 2 of the 5 metrics used anyway). Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: CASSANDRA-8662v1.txt cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/3] cassandra git commit: Add tooling to detect hot partitions
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 1435b9a87 - faf91818b
refs/heads/trunk 184bb65fc - 0a09b87dc
Add tooling to detect hot partitions
Patch by Chris Lohfink, reviewed by brandonwilliams for CASSANDRA-7974
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/faf91818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/faf91818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/faf91818
Branch: refs/heads/cassandra-2.1
Commit: faf91818b46fb51ed576664a1119315e7b7c3383
Parents: 1435b9a
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jan 21 11:45:45 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jan 21 11:45:45 2015 -0600
--
CHANGES.txt | 1 +
.../apache/cassandra/db/ColumnFamilyStore.java | 69 -
.../cassandra/db/ColumnFamilyStoreMBean.java| 14 ++
.../cassandra/metrics/ColumnFamilyMetrics.java | 19 ++-
.../org/apache/cassandra/tools/NodeProbe.java | 26 +++-
.../org/apache/cassandra/tools/NodeTool.java| 83 ++-
.../org/apache/cassandra/utils/TopKSampler.java | 139 ++
.../apache/cassandra/utils/TopKSamplerTest.java | 147 +++
8 files changed, 482 insertions(+), 16 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/faf91818/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0c2bab8..f1eaa77 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Add tooling to detect hot partitions (CASSANDRA-7974)
* Fix cassandra-stress user-mode truncation of partition generation
(CASSANDRA-8608)
* Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
* Don't allow starting multiple inc repairs on the same sstables
(CASSANDRA-8316)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/faf91818/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index f7a691e..0c95b0e 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -26,14 +26,12 @@ import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;
import java.util.regex.Pattern;
import javax.management.*;
+import javax.management.openmbean.*;
import com.google.common.annotations.VisibleForTesting;
-import com.google.common.base.Function;
-import com.google.common.base.Predicate;
+import com.google.common.base.*;
import com.google.common.collect.*;
import com.google.common.util.concurrent.*;
-import com.google.common.util.concurrent.Futures;
-import com.google.common.util.concurrent.Uninterruptibles;
import org.json.simple.*;
import org.slf4j.Logger;
@@ -68,14 +66,18 @@ import
org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataType;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.metrics.ColumnFamilyMetrics;
+import org.apache.cassandra.metrics.ColumnFamilyMetrics.Sampler;
import org.apache.cassandra.service.CacheService;
import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.streaming.StreamLockfile;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.TopKSampler.SamplerResult;
import org.apache.cassandra.utils.concurrent.OpOrder;
import org.apache.cassandra.utils.memory.MemtableAllocator;
+import com.clearspring.analytics.stream.Counter;
+
public class ColumnFamilyStore implements ColumnFamilyStoreMBean
{
private static final Logger logger =
LoggerFactory.getLogger(ColumnFamilyStore.class);
@@ -102,6 +104,39 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
new NamedThreadFactory(MemtableReclaimMemory),
internal);
+private static final String[] COUNTER_NAMES = new String[]{raw, count,
error, string};
+private static final String[] COUNTER_DESCS = new String[]
+{ partition key in raw hex bytes,
+ value of this partition for given sampler,
+ value is within the error bounds plus or minus of this,
+ the partition key turned into a human readable format };
+private static final CompositeType COUNTER_COMPOSITE_TYPE;
+private static final TabularType
[jira] [Updated] (CASSANDRA-7974) Enable tooling to detect hot partitions
[ https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7974: - Attachment: CASSANDRA-7974v5.txt Fixed in v5 an issue - the byte array it was using was not checked for equality in the StreamSummary so the same key may show up as duplicates. Enable tooling to detect hot partitions --- Key: CASSANDRA-7974 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: 7974.txt, CASSANDRA-7974v3.txt, CASSANDRA-7974v4.txt, CASSANDRA-7974v5.txt, cassandra-2.1-7974v2.txt Sometimes you know you have a hot partition by the load on a replica set, but have no way of determining which partition it is. Tracing is inadequate for this without a lot of post-tracing analysis that might not yield results. Since we already include stream-lib for HLL in compaction metadata, it shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then return the top partitions hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
[
https://issues.apache.org/jira/browse/CASSANDRA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286130#comment-14286130
]
Andrew J Lenards edited comment on CASSANDRA-8663 at 1/21/15 7:34 PM:
--
I should also note that doing the same operations in cqlsh for (DSE 4.6.0 /
Cassandra 2.0.11.83) does not yield the integer display behavior.
{noformat}
cqlsh:system CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh:system CONSISTENCY;
Current consistency level is QUORUM.
cqlsh:system CONSISTENCY EACH_QUORUM;
Consistency level set to EACH_QUORUM.
{noformat}
was (Author: alenards):
I should also note that doing the same operations in cqlsh for (DSE 4.6.0 /
Cassandra 2.0.11.83) does not yield the integer behavior.
{noformat}
cqlsh:system CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh:system CONSISTENCY;
Current consistency level is QUORUM.
cqlsh:system CONSISTENCY EACH_QUORUM;
Consistency level set to EACH_QUORUM.
{noformat}
cqlsh is showing the integer value for ConsistencyLevel not name
-
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
Labels: lhf
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The
name of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Serializing Row cache alternative, fully off heap
Repository: cassandra Updated Branches: refs/heads/trunk 732986bbd - dfd78d0e9 Serializing Row cache alternative, fully off heap Patch by Robert Stupp, reviewed by Ariel Weisberg for CASSANDRA-7438 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dfd78d0e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dfd78d0e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dfd78d0e Branch: refs/heads/trunk Commit: dfd78d0e90b9995872b00f6f33afc180f2b7cba1 Parents: 732986b Author: Robert Stupp sn...@snazy.de Authored: Wed Jan 21 20:50:35 2015 +0100 Committer: Robert Stupp sn...@snazy.de Committed: Wed Jan 21 20:50:35 2015 +0100 -- CHANGES.txt | 1 + NOTICE.txt | 5 + build.xml | 3 +- conf/cassandra.yaml | 22 +- lib/licenses/ohc-0.2.1.txt | 201 ++ lib/ohc-core-0.3.1.jar | Bin 0 - 143623 bytes lib/ohc-core-j8-0.3.1.jar | Bin 0 - 5026 bytes .../apache/cassandra/cache/AutoSavingCache.java | 42 ++- .../apache/cassandra/cache/CacheProvider.java | 23 ++ .../cache/ConcurrentLinkedHashCache.java| 12 +- src/java/org/apache/cassandra/cache/ICache.java | 6 +- .../cassandra/cache/InstrumentingCache.java | 24 +- .../cassandra/cache/NopCacheProvider.java | 93 +++ .../org/apache/cassandra/cache/OHCProvider.java | 274 +++ .../org/apache/cassandra/cache/RowCacheKey.java | 6 + .../cassandra/cache/SerializingCache.java | 10 +- .../cache/SerializingCacheProvider.java | 7 +- .../org/apache/cassandra/config/Config.java | 1 + .../cassandra/config/DatabaseDescriptor.java| 15 +- .../config/YamlConfigurationLoader.java | 2 +- .../apache/cassandra/db/ColumnFamilyStore.java | 12 +- .../apache/cassandra/service/CacheService.java | 23 +- test/conf/cassandra.yaml| 2 + .../org/apache/cassandra/db/KeyCacheTest.java | 9 +- .../org/apache/cassandra/db/RowCacheTest.java | 14 +- 25 files changed, 731 insertions(+), 76 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dfd78d0e/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index cc6478e..99a5d51 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0 + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) * Support direct buffer decompression for reads (CASSANDRA-8464) http://git-wip-us.apache.org/repos/asf/cassandra/blob/dfd78d0e/NOTICE.txt -- diff --git a/NOTICE.txt b/NOTICE.txt index 2fe15f5..43d4514 100644 --- a/NOTICE.txt +++ b/NOTICE.txt @@ -66,3 +66,8 @@ Javassist SIGAR http://sigar.hyperic.com/ + +OHC +(https://github.com/snazy/ohc) +Java Off-Heap-Cache, licensed under APLv2 +Copyright 2014-2015 Robert Stupp, Germany. http://git-wip-us.apache.org/repos/asf/cassandra/blob/dfd78d0e/build.xml -- diff --git a/build.xml b/build.xml index baf6a77..fccd009 100644 --- a/build.xml +++ b/build.xml @@ -375,6 +375,7 @@ dependency groupId=com.clearspring.analytics artifactId=stream version=2.5.2 / dependency groupId=com.datastax.cassandra artifactId=cassandra-driver-core version=2.1.2 / dependency groupId=org.javassist artifactId=javassist version=3.18.2-GA / + dependency groupId=org.caffinitas.ohc artifactId=ohc-core version=0.3.1 / dependency groupId=net.sf.supercsv artifactId=super-csv version=2.1.0 / dependency groupId=net.ju-n.compile-command-annotations artifactId=compile-command-annotations version=1.2.0 / dependency groupId=org.fusesource artifactId=sigar version=1.6.4 @@ -424,10 +425,10 @@ dependency groupId=org.antlr artifactId=antlr/ dependency groupId=com.datastax.cassandra artifactId=cassandra-driver-core/ dependency groupId=org.javassist artifactId=javassist/ +dependency groupId=org.caffinitas.ohc artifactId=ohc-core/ dependency groupId=org.openjdk.jmh artifactId=jmh-core/ dependency groupId=org.openjdk.jmh artifactId=jmh-generator-annprocess/ dependency groupId=net.ju-n.compile-command-annotations artifactId=compile-command-annotations/ -dependency groupId=org.javassist artifactId=javassist / /artifact:pom
[jira] [Updated] (CASSANDRA-8616) sstable2json may result in commit log segments be written
[ https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russ Hatch updated CASSANDRA-8616: -- Tester: Russ Hatch sstable2json may result in commit log segments be written - Key: CASSANDRA-8616 URL: https://issues.apache.org/jira/browse/CASSANDRA-8616 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Tyler Hobbs Assignee: Yuki Morishita Fix For: 2.0.13 Attachments: 8161-2.0.txt There was a report of sstable2json causing commitlog segments to be written out when run. I haven't attempted to reproduce this yet, so that's all I know for now. Since sstable2json loads the conf and schema, I'm thinking that it may inadvertently be triggering the commitlog code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/tools/NodeTool.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0a09b87d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0a09b87d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0a09b87d Branch: refs/heads/trunk Commit: 0a09b87dcefee496dbfa6a398fb18033e32042d5 Parents: 184bb65 faf9181 Author: Brandon Williams brandonwilli...@apache.org Authored: Wed Jan 21 11:49:08 2015 -0600 Committer: Brandon Williams brandonwilli...@apache.org Committed: Wed Jan 21 11:49:08 2015 -0600 -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 69 - .../cassandra/db/ColumnFamilyStoreMBean.java| 14 ++ .../cassandra/metrics/ColumnFamilyMetrics.java | 19 ++- .../org/apache/cassandra/tools/NodeProbe.java | 26 +++- .../org/apache/cassandra/tools/NodeTool.java| 85 ++- .../org/apache/cassandra/utils/TopKSampler.java | 139 ++ .../apache/cassandra/utils/TopKSamplerTest.java | 147 +++ 8 files changed, 483 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a09b87d/CHANGES.txt -- diff --cc CHANGES.txt index 66b17ce,f1eaa77..9343d66 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,58 -1,5 +1,59 @@@ +3.0 + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Add role based access control (CASSANDRA-7653) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * Improve concurrency of repair (CASSANDRA-6455, 8208) + + 2.1.3 + * Add tooling to detect hot partitions (CASSANDRA-7974) * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) * Don't allow starting multiple inc repairs on the same sstables
[jira] [Commented] (CASSANDRA-8660) Cassandra makes Java 1.7+ SEGFAULT
[ https://issues.apache.org/jira/browse/CASSANDRA-8660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285929#comment-14285929 ] Pierre Belanger commented on CASSANDRA-8660: About 600G / node Cassandra makes Java 1.7+ SEGFAULT -- Key: CASSANDRA-8660 URL: https://issues.apache.org/jira/browse/CASSANDRA-8660 Project: Cassandra Issue Type: Bug Components: Core Environment: Linux Redhat 6.5 Reporter: Pierre Belanger Priority: Critical Attachments: 8660, hs_err_pid12004.log We upgraded a 3 nodes cluster from Cassandra 2.0.11-5f54285e9e to 2.1-7e6d9eb842. With Cassandra 2.1 , within ~10 min. cassandra manages to makes Java 1.7.0_72-b14 segfault. This is also tested and same behavior w/ Java 1.7.0_60-b19 and 1.8.0_25-b17. We were able to use same data, downgrade back to 2.0 and things ran for ~3-4 days w/ no hick up. Upgrading back to Cassandra-2.1, Java does a segfault again. See attached hs_err file. Pierre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
[
https://issues.apache.org/jira/browse/CASSANDRA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew J Lenards updated CASSANDRA-8663:
Description:
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The name
of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
was:
When you use issues the ``CONSISTENCY;`` command in `cqlsh`, it is showing the
integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The name
of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
cqlsh is showing the integer value for ConsistencyLevel not name
-
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The
name of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286143#comment-14286143
]
Philip Thompson commented on CASSANDRA-8061:
//cc [~benedict]
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
113M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
116M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Data.db
712K
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286125#comment-14286125
]
Mark Cooper commented on CASSANDRA-8061:
Thanks for merging this in (and marking this bug as critical)! I will give
this a try either Friday or early next week. To be clear I should pull down
this release and build it:
https://github.com/belliottsmith/cassandra/tree/7705-2.1
Any other instructions? Something I should look for the logs?
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
113M
[jira] [Updated] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-8662: - Issue Type: Improvement (was: Bug) Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-8514:
--
Attachment: CASSANDRA-8514-V2.txt
I do not think that the fact that {{combineHistograms}} returns an empty array
is bad. I find it more natural than returning {{null}}. Both cases would have
broke {{NodeTool}} anyway.
The patch checks the values returned by {{NodeProbe}} and print an error
message if no SSTables exist. The missing value are replaced by {{Double.NaN}}
as it is the case for the other error cases.
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Benjamin Lerer
Fix For: 2.1.3
Attachments: CASSANDRA-8514-V2.txt, cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8437) Track digest mismatch ratio
[ https://issues.apache.org/jira/browse/CASSANDRA-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285814#comment-14285814 ] Benjamin Lerer commented on CASSANDRA-8437: --- I agree, I will change that. Track digest mismatch ratio --- Key: CASSANDRA-8437 URL: https://issues.apache.org/jira/browse/CASSANDRA-8437 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Assignee: Benjamin Lerer Priority: Minor Fix For: 2.1.3 Attachments: CASSANDRA-8437.txt I don't believe we track how often read results in a digest mismatch but we should since that could directly impact read performance in practice. Once we have that data, it might be that some workloads (write heavy most likely) ends up with enough mismatches that going to the data read is more efficient in practice. What we do about it it step 2 however, but getting the data is easy enough. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Fix cassandra-stress user-mode truncation of partition generation
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 576a75f28 - 1435b9a87
Fix cassandra-stress user-mode truncation of partition generation
patch by benedict; reviewed by tjake for CASSANDRA-8608
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1435b9a8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1435b9a8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1435b9a8
Branch: refs/heads/cassandra-2.1
Commit: 1435b9a87a4e1878cb35bd6d75e631bf2093e460
Parents: 576a75f
Author: Benedict Elliott Smith bened...@apache.org
Authored: Wed Jan 21 16:50:55 2015 +
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Wed Jan 21 16:50:55 2015 +
--
CHANGES.txt | 1 +
.../org/apache/cassandra/stress/Operation.java | 4 +-
.../stress/generate/PartitionIterator.java | 219 ++-
3 files changed, 122 insertions(+), 102 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1435b9a8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8f71269..0c2bab8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Fix cassandra-stress user-mode truncation of partition generation
(CASSANDRA-8608)
* Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
* Don't allow starting multiple inc repairs on the same sstables
(CASSANDRA-8316)
* Invalidate prepared BATCH statements when related tables
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1435b9a8/tools/stress/src/org/apache/cassandra/stress/Operation.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/Operation.java
b/tools/stress/src/org/apache/cassandra/stress/Operation.java
index edf3a54..05045f8 100644
--- a/tools/stress/src/org/apache/cassandra/stress/Operation.java
+++ b/tools/stress/src/org/apache/cassandra/stress/Operation.java
@@ -105,9 +105,9 @@ public abstract class Operation
break;
if (spec.useRatio == null)
-success = partitionCache.get(i).reset(seed,
spec.targetCount, this);
+success = partitionCache.get(i).reset(seed,
spec.targetCount, isWrite());
else
-success = partitionCache.get(i).reset(seed,
spec.useRatio.next(), this);
+success = partitionCache.get(i).reset(seed,
spec.useRatio.next(), isWrite());
}
}
partitionCount = i;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1435b9a8/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
b/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
index 0d0cba1..0466edb 100644
---
a/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
+++
b/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
@@ -50,14 +50,16 @@ import
org.apache.cassandra.stress.generate.values.Generator;
public abstract class PartitionIterator implements IteratorRow
{
-// we reuse the row object to save garbage
-abstract boolean reset(double useChance, int targetCount, Operation op);
+abstract boolean reset(double useChance, int targetCount, boolean isWrite);
long idseed;
Seed seed;
-final Object[] partitionKey;
+
final PartitionGenerator generator;
final SeedManager seedManager;
+
+// we reuse these objects to save garbage
+final Object[] partitionKey;
final Row row;
public static PartitionIterator get(PartitionGenerator generator,
SeedManager seedManager)
@@ -93,16 +95,16 @@ public abstract class PartitionIterator implements
IteratorRow
this.idseed = idseed;
}
-public boolean reset(Seed seed, double useChance, Operation op)
+public boolean reset(Seed seed, double useChance, boolean isWrite)
{
setSeed(seed);
-return reset(useChance, 0, op);
+return reset(useChance, 0, isWrite);
}
-public boolean reset(Seed seed, int targetCount, Operation op)
+public boolean reset(Seed seed, int targetCount, boolean isWrite)
{
setSeed(seed);
-return reset(Double.NaN, targetCount, op);
+return reset(Double.NaN, targetCount, isWrite);
}
static class SingleRowIterator extends PartitionIterator
@@ -115,10 +117,10 @@ public abstract class PartitionIterator implements
IteratorRow
super(generator, seedManager);
[jira] [Updated] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8662: -- Fix Version/s: 2.1.3 Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink Fix For: 2.1.3 cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285799#comment-14285799
]
Jimmy Mårdell commented on CASSANDRA-8514:
--
No problem [~blerer], feel free to work on it. I just submitted a quick patch
but as [~jbellis] points out, it's better to fix the underlying problem.
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Jimmy Mårdell
Fix For: 2.1.3
Attachments: cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8651) Add support for running on Apache Mesos
[ https://issues.apache.org/jira/browse/CASSANDRA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-8651: Priority: Minor (was: Major) Add support for running on Apache Mesos --- Key: CASSANDRA-8651 URL: https://issues.apache.org/jira/browse/CASSANDRA-8651 Project: Cassandra Issue Type: Task Reporter: Ben Whitehead Priority: Minor Fix For: 3.0 As a user of Apache Mesos I would like to be able to run Cassandra on my Mesos cluster. This would entail integration of Cassandra on Mesos through the creation of a production level Mesos framework. This would enable me to avoid static partitioning and inefficiencies and run Cassandra as part of my data center infrastructure. http://mesos.apache.org/documentation/latest/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7974) Enable tooling to detect hot partitions
[ https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7974: - Attachment: CASSANDRA-7974v4.txt Enable tooling to detect hot partitions --- Key: CASSANDRA-7974 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Chris Lohfink Fix For: 2.1.3 Attachments: 7974.txt, CASSANDRA-7974v3.txt, CASSANDRA-7974v4.txt, cassandra-2.1-7974v2.txt Sometimes you know you have a hot partition by the load on a replica set, but have no way of determining which partition it is. Tracing is inadequate for this without a lot of post-tracing analysis that might not yield results. Since we already include stream-lib for HLL in compaction metadata, it shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then return the top partitions hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8662) Switch cfhistograms from using yammer metrics in 2.1
Chris Lohfink created CASSANDRA-8662: Summary: Switch cfhistograms from using yammer metrics in 2.1 Key: CASSANDRA-8662 URL: https://issues.apache.org/jira/browse/CASSANDRA-8662 Project: Cassandra Issue Type: Bug Components: Core Reporter: Chris Lohfink Assignee: Chris Lohfink cfhistograms was changed to use the yammer histograms CASSANDRA-5871. Until CASSANDRA-5657's change to metrics we shouldn't represent the yammer histogram for latency metrics. Yammer's histogram is based on assumption of a normal distribution and is bad at representing the tail percentiles with skewed distributions. So revert the change just in the 2.1 branch but leave it for 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8437) Track digest mismatch ratio
[ https://issues.apache.org/jira/browse/CASSANDRA-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285810#comment-14285810 ] Brandon Williams commented on CASSANDRA-8437: - Sure, and I'm fine with that, just pointing out that putting this in info instead of netstats at this point makes it the odd man out. Track digest mismatch ratio --- Key: CASSANDRA-8437 URL: https://issues.apache.org/jira/browse/CASSANDRA-8437 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Assignee: Benjamin Lerer Priority: Minor Fix For: 2.1.3 Attachments: CASSANDRA-8437.txt I don't believe we track how often read results in a digest mismatch but we should since that could directly impact read performance in practice. Once we have that data, it might be that some workloads (write heavy most likely) ends up with enough mismatches that going to the data read is more efficient in practice. What we do about it it step 2 however, but getting the data is easy enough. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285795#comment-14285795
]
Benjamin Lerer edited comment on CASSANDRA-8514 at 1/21/15 3:46 PM:
Sorry [~yarin]. I did not realized that you had started to work on that ticket.
was (Author: blerer):
Sorry [~yarin] did not realized that you had started to work on that ticket.
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Jimmy Mårdell
Fix For: 2.1.3
Attachments: cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer reassigned CASSANDRA-8514:
-
Assignee: Benjamin Lerer (was: Jimmy Mårdell)
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Benjamin Lerer
Fix For: 2.1.3
Attachments: cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8558) deleted row still can be selected out
[
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8558:
---
Labels: qa-resolved (was: )
deleted row still can be selected out
-
Key: CASSANDRA-8558
URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 2.1.2
java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
Labels: qa-resolved
Fix For: 2.0.12, 2.1.3
Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt
first
{code}CREATE KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
CREATE TABLE space1.table3(a int, b int, c text,primary key(a,b));
CREATE KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};{code}
second
{code}CREATE TABLE space2.table1(a int, b int, c int, primary key(a,b));
CREATE TABLE space2.table2(a int, b int, c int, primary key(a,b));
INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
drop table space2.table1;
DELETE FROM space1.table3 where a=1 and b=1;
drop table space2.table2;
select * from space1.table3 where a=1 and b=1;{code}
you will find that the row (a=1 and b=1) in space1.table3 is not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[1/2] cassandra git commit: Fix cassandra-stress user-mode truncation of partition generation
Repository: cassandra
Updated Branches:
refs/heads/trunk 37f127897 - 184bb65fc
Fix cassandra-stress user-mode truncation of partition generation
patch by benedict; reviewed by tjake for CASSANDRA-8608
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1435b9a8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1435b9a8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1435b9a8
Branch: refs/heads/trunk
Commit: 1435b9a87a4e1878cb35bd6d75e631bf2093e460
Parents: 576a75f
Author: Benedict Elliott Smith bened...@apache.org
Authored: Wed Jan 21 16:50:55 2015 +
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Wed Jan 21 16:50:55 2015 +
--
CHANGES.txt | 1 +
.../org/apache/cassandra/stress/Operation.java | 4 +-
.../stress/generate/PartitionIterator.java | 219 ++-
3 files changed, 122 insertions(+), 102 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1435b9a8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8f71269..0c2bab8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Fix cassandra-stress user-mode truncation of partition generation
(CASSANDRA-8608)
* Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
* Don't allow starting multiple inc repairs on the same sstables
(CASSANDRA-8316)
* Invalidate prepared BATCH statements when related tables
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1435b9a8/tools/stress/src/org/apache/cassandra/stress/Operation.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/Operation.java
b/tools/stress/src/org/apache/cassandra/stress/Operation.java
index edf3a54..05045f8 100644
--- a/tools/stress/src/org/apache/cassandra/stress/Operation.java
+++ b/tools/stress/src/org/apache/cassandra/stress/Operation.java
@@ -105,9 +105,9 @@ public abstract class Operation
break;
if (spec.useRatio == null)
-success = partitionCache.get(i).reset(seed,
spec.targetCount, this);
+success = partitionCache.get(i).reset(seed,
spec.targetCount, isWrite());
else
-success = partitionCache.get(i).reset(seed,
spec.useRatio.next(), this);
+success = partitionCache.get(i).reset(seed,
spec.useRatio.next(), isWrite());
}
}
partitionCount = i;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1435b9a8/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
b/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
index 0d0cba1..0466edb 100644
---
a/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
+++
b/tools/stress/src/org/apache/cassandra/stress/generate/PartitionIterator.java
@@ -50,14 +50,16 @@ import
org.apache.cassandra.stress.generate.values.Generator;
public abstract class PartitionIterator implements IteratorRow
{
-// we reuse the row object to save garbage
-abstract boolean reset(double useChance, int targetCount, Operation op);
+abstract boolean reset(double useChance, int targetCount, boolean isWrite);
long idseed;
Seed seed;
-final Object[] partitionKey;
+
final PartitionGenerator generator;
final SeedManager seedManager;
+
+// we reuse these objects to save garbage
+final Object[] partitionKey;
final Row row;
public static PartitionIterator get(PartitionGenerator generator,
SeedManager seedManager)
@@ -93,16 +95,16 @@ public abstract class PartitionIterator implements
IteratorRow
this.idseed = idseed;
}
-public boolean reset(Seed seed, double useChance, Operation op)
+public boolean reset(Seed seed, double useChance, boolean isWrite)
{
setSeed(seed);
-return reset(useChance, 0, op);
+return reset(useChance, 0, isWrite);
}
-public boolean reset(Seed seed, int targetCount, Operation op)
+public boolean reset(Seed seed, int targetCount, boolean isWrite)
{
setSeed(seed);
-return reset(Double.NaN, targetCount, op);
+return reset(Double.NaN, targetCount, isWrite);
}
static class SingleRowIterator extends PartitionIterator
@@ -115,10 +117,10 @@ public abstract class PartitionIterator implements
IteratorRow
super(generator, seedManager);
}
-
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/184bb65f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/184bb65f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/184bb65f Branch: refs/heads/trunk Commit: 184bb65fcafb8603840fbbbda59665dcda275da3 Parents: 37f1278 1435b9a Author: Benedict Elliott Smith bened...@apache.org Authored: Wed Jan 21 16:52:20 2015 + Committer: Benedict Elliott Smith bened...@apache.org Committed: Wed Jan 21 16:52:20 2015 + -- CHANGES.txt | 2 + .../org/apache/cassandra/stress/Operation.java | 4 +- .../stress/generate/PartitionIterator.java | 219 ++- 3 files changed, 123 insertions(+), 102 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/184bb65f/CHANGES.txt -- diff --cc CHANGES.txt index 6d60699,0c2bab8..66b17ce --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,58 -1,6 +1,60 @@@ +3.0 + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Add role based access control (CASSANDRA-7653) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * Improve concurrency of repair (CASSANDRA-6455, 8208) + + 2.1.3 + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) * Invalidate prepared BATCH statements when related tables or keyspaces are dropped (CASSANDRA-8652)
[jira] [Updated] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-8514:
--
Assignee: Jimmy Mårdell (was: Benjamin Lerer)
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Jimmy Mårdell
Fix For: 2.1.3
Attachments: cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285795#comment-14285795
]
Benjamin Lerer commented on CASSANDRA-8514:
---
Sorry [~yarin] did not realized that you had started to work on that ticket.
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Jimmy Mårdell
Fix For: 2.1.3
Attachments: cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8514:
---
Tester: Philip Thompson
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Benjamin Lerer
Fix For: 2.1.3
Attachments: CASSANDRA-8514-V2.txt, cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285875#comment-14285875
]
Benjamin Lerer commented on CASSANDRA-8514:
---
[~jbellis] do you want to be the reviewer?
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Benjamin Lerer
Fix For: 2.1.3
Attachments: CASSANDRA-8514-V2.txt, cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: ninja - fix flappy unit tests in UFTest that require native protocol server socket
Repository: cassandra
Updated Branches:
refs/heads/trunk dfd78d0e9 - 39108df7c
ninja - fix flappy unit tests in UFTest that require native protocol server
socket
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/39108df7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/39108df7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/39108df7
Branch: refs/heads/trunk
Commit: 39108df7c8e0ffae3b39ad4adfefcd18f38667f7
Parents: dfd78d0
Author: Robert Stupp sn...@snazy.de
Authored: Wed Jan 21 22:10:40 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Wed Jan 21 22:10:40 2015 +0100
--
test/unit/org/apache/cassandra/cql3/CQLTester.java | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/39108df7/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java
b/test/unit/org/apache/cassandra/cql3/CQLTester.java
index b01ca9e..efda704 100644
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@ -18,7 +18,6 @@
package org.apache.cassandra.cql3;
import java.io.File;
-import java.io.IOException;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.net.InetAddress;
@@ -89,10 +88,13 @@ public abstract class CQLTester
try
{
-ServerSocket serverSocket = new ServerSocket(0);
-nativePort = serverSocket.getLocalPort();
+try (ServerSocket serverSocket = new ServerSocket(0))
+{
+nativePort = serverSocket.getLocalPort();
+}
+Thread.sleep(250);
}
-catch (IOException e)
+catch (Exception e)
{
throw new RuntimeException(e);
}
[jira] [Resolved] (CASSANDRA-6544) Reduce GC activity during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-6544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-6544. --- Resolution: Later Reviewer: (was: Pavel Yaskevich) Going to resolve as Later since this probably isn't the top concern in 2.1+ anymore. Reduce GC activity during compaction Key: CASSANDRA-6544 URL: https://issues.apache.org/jira/browse/CASSANDRA-6544 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Vijay Labels: compaction Fix For: 2.1.3 We are noticing increase in P99 while the compactions are running at full stream. Most of it is because of the increased GC activity (followed by full GC). The obvious solution/work around is to throttle the compactions, but with SSD's we can get more disk bandwidth for reads and compactions. It will be nice to move the compaction object allocations off heap. First thing to do might be create a Offheap Slab allocator with the size as the compaction in memory size and recycle it. Also we might want to make it configurable so folks can disable it when they don't have off-heap memory to reserve. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
[
https://issues.apache.org/jira/browse/CASSANDRA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson resolved CASSANDRA-8663.
Resolution: Duplicate
cqlsh is showing the integer value for ConsistencyLevel not name
-
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
Labels: lhf
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The
name of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8663) cqlsh is showing the integer value for ConsistencyLevel not name
[
https://issues.apache.org/jira/browse/CASSANDRA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8663:
---
Fix Version/s: 2.1.3
Labels: cqlsh (was: lhf)
cqlsh is showing the integer value for ConsistencyLevel not name
-
Key: CASSANDRA-8663
URL: https://issues.apache.org/jira/browse/CASSANDRA-8663
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Cassandra 2.1.2-SNAPSHOT, built/fetched with ccm
Reporter: Andrew J Lenards
Priority: Minor
Labels: cqlsh
Fix For: 2.1.3
When you use issues the {{CONSISTENCY;}} command in {{cqlsh}}, it is showing
the integer value for the current consistency level, not the name of that
consistency level.
Example:
{noformat}
cqlsh CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh CONSISTENCY;
Current consistency level is 4.
{noformat}
The output can be confusing for some who might think that it refers to the
number of nodes required for acknowledge when performing an operation. The
name of the current consistency level would be more clear.
More examples:
https://gist.github.com/lenards/e97ce13347e7ad454b86
Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8308) Windows: Commitlog access violations on unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp resolved CASSANDRA-8308.
-
Resolution: Fixed
Windows: Commitlog access violations on unit tests
--
Key: CASSANDRA-8308
URL: https://issues.apache.org/jira/browse/CASSANDRA-8308
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Labels: Windows
Fix For: 3.0
Attachments: 8308_v1.txt, 8308_v2.txt, 8308_v3.txt
We have four unit tests failing on trunk on Windows, all with
FileSystemException's related to the SchemaLoader:
{noformat}
[junit] Test
org.apache.cassandra.db.compaction.DateTieredCompactionStrategyTest FAILED
[junit] Test org.apache.cassandra.cql3.ThriftCompatibilityTest FAILED
[junit] Test org.apache.cassandra.io.sstable.SSTableRewriterTest FAILED
[junit] Test org.apache.cassandra.repair.LocalSyncTaskTest FAILED
{noformat}
Example error:
{noformat}
[junit] Caused by: java.nio.file.FileSystemException:
build\test\cassandra\commitlog;0\CommitLog-5-1415908745965.log: The process
cannot access the file because it is being used by another process.
[junit]
[junit] at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[junit] at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[junit] at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[junit] at java.nio.file.Files.delete(Files.java:1079)
[junit] at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:125)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8651) Add support for running on Apache Mesos
[ https://issues.apache.org/jira/browse/CASSANDRA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286428#comment-14286428 ] Jonathan Ellis commented on CASSANDRA-8651: --- bq. As a user of Apache Mesos I would like to be able to run Cassandra on my Mesos cluster What is stopping you? Is this something C* needs to write code to support? Add support for running on Apache Mesos --- Key: CASSANDRA-8651 URL: https://issues.apache.org/jira/browse/CASSANDRA-8651 Project: Cassandra Issue Type: Task Reporter: Ben Whitehead Priority: Minor Fix For: 3.0 As a user of Apache Mesos I would like to be able to run Cassandra on my Mesos cluster. This would entail integration of Cassandra on Mesos through the creation of a production level Mesos framework. This would enable me to avoid static partitioning and inefficiencies and run Cassandra as part of my data center infrastructure. http://mesos.apache.org/documentation/latest/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8308) Windows: Commitlog access violations on unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286434#comment-14286434
]
Robert Stupp commented on CASSANDRA-8308:
-
I tried on my Linux box and on another (5 yrs old) MBP: tests pass.
[~iamaleksey]'s Mac does also have no timeouts.
Seems to be a pure local (mine) problem. Sorry for alerting for nothing.
Windows: Commitlog access violations on unit tests
--
Key: CASSANDRA-8308
URL: https://issues.apache.org/jira/browse/CASSANDRA-8308
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Labels: Windows
Fix For: 3.0
Attachments: 8308_v1.txt, 8308_v2.txt, 8308_v3.txt
We have four unit tests failing on trunk on Windows, all with
FileSystemException's related to the SchemaLoader:
{noformat}
[junit] Test
org.apache.cassandra.db.compaction.DateTieredCompactionStrategyTest FAILED
[junit] Test org.apache.cassandra.cql3.ThriftCompatibilityTest FAILED
[junit] Test org.apache.cassandra.io.sstable.SSTableRewriterTest FAILED
[junit] Test org.apache.cassandra.repair.LocalSyncTaskTest FAILED
{noformat}
Example error:
{noformat}
[junit] Caused by: java.nio.file.FileSystemException:
build\test\cassandra\commitlog;0\CommitLog-5-1415908745965.log: The process
cannot access the file because it is being used by another process.
[junit]
[junit] at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[junit] at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[junit] at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[junit] at java.nio.file.Files.delete(Files.java:1079)
[junit] at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:125)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8649) CAS per (partition key + clustering key) and not only per (partition key)
[
https://issues.apache.org/jira/browse/CASSANDRA-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-8649.
---
Resolution: Later
3 wontfix and 1 Later, closing as Later.
(I'm sympathetic to this will confuse SQL users arguments, but unlike almost
all RDBMS, C* is a distributed system, and the partition is the unit of
replication. Once you understand that, it is intuitive to have that the
granularity of LWT.)
CAS per (partition key + clustering key) and not only per (partition key)
-
Key: CASSANDRA-8649
URL: https://issues.apache.org/jira/browse/CASSANDRA-8649
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Dobrin
Fix For: 3.0
Reading the description at
http://www.datastax.com/dev/blog/lightweight-transactions-in-cassandra-2-0 :
...
* The columns updated do NOT have to be the same as the columns in the IF
clause.
* Lightweight transactions are restricted to a single partition; this is the
granularity at which we keep the internal Paxos state. As a corollary,
transactions in different partitions will never interrupt each other.
...
So my understanding of the above is that if multiple writers for example
perform CAS inserts (INSERT...IF NOT EXISTS) using the same partition key and
different clustering keys will interrupt/interfere with each other?
Is this understanding correct? (my tests seems to confirm it)
For example if I want to model users from different country/city/area and I
want to be able to list all the users from a given country ordered by
(city,area) and also I know that a single cassandra node will be able to
store all the users from a given country but I need to partition users from
different countries because a single cassandra node will not be enough:
CREATE COLUMNFAMILY user (
country text,
city text,
area text,
id text,
json text,
version bigint,
PRIMARY KEY ((country), city, area, id)
);
Where id is the user id and json is a JSON serialized user object (an
aggregate) containing more information about the user.
I want to be able to CAS insert many users into the same country concurrently
using
INSERT INTO user(country, city, area, id, json, version) VALUES
('x',...) IF NOT EXISTS;
and be able to CAS update users from the same country concurrently:
UPDATE user SET json='{...}',version=18 WHERE country='x' AND city='y'
AND area='z' AND id='123' IF version=17;
As I understand this will not be efficient because all the above concurrent
statements will have to be ordered by the same paxos instance/state per
country 'x'? (and trying it results in a lot of WriteTimeoutException-s)
If yes - can we made paxos to support IF statements per column/cell?
By cell/column I mean all the underlying persistent state that is behind the
compound primary key (partition key + clustering key) - in the above example
the state is json and version
the partition key is the country
and the clustering key is (city, area, id)
(
I'm stating it explicitly as I'm not completely sure whether this is a single
cell or double cells underneath at the storage engine, references used:
http://www.datastax.com/dev/blog/cql3-for-cassandra-experts
http://www.datastax.com/dev/blog/does-cql-support-dynamic-columns-wide-rows
)
In other words is it possible to make CAS per (partition key + clustering
key) and not only per (partition key)?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8623) sstablesplit fails *randomly* with Data component is missing
[
https://issues.apache.org/jira/browse/CASSANDRA-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286409#comment-14286409
]
Jonathan Ellis commented on CASSANDRA-8623:
---
/cc [~benedict]
sstablesplit fails *randomly* with Data component is missing
Key: CASSANDRA-8623
URL: https://issues.apache.org/jira/browse/CASSANDRA-8623
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
Assignee: Marcus Eriksson
Attachments: output.log
I'm experiencing an issue related to sstablesplit. I would like to understand
if I am doing something wrong or there is an issue in the split process. The
process fails randomly with the following exception:
{code}
ERROR 02:17:36 Error in ThreadPoolExecutor
java.lang.AssertionError: Data component is missing for
sstable./tools/bin/../../data/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-16
{code}
See attached output.log file. The process never stops after this exception
and I've also seen the dataset growing indefinitely (number of sstables).
* I have not been able to reproduce the issue with a single sstablesplit
command. ie, specifying all files with glob matching.
* I can reproduce the bug if I call multiple sstablesplit one file at the
time (the way ccm does)
Here is the test case file to reproduce the bug:
https://drive.google.com/file/d/0BwZ_GPM33j6KdVh0NTdkOWV2R1E/view?usp=sharing
1. Download the split_issue.tar.gz file. It includes latest cassandra-2.1
branch binaries.
2. Extract it
3. CD inside the use case directory
4. Download the dataset (2G) just to be sure we have the same thing, and
place it in the working directory.
https://docs.google.com/uc?id=0BwZ_GPM33j6KV3ViNnpPcVFndUUexport=download
5. The first time, run ./test.sh. This will setup and run a test.
6. The next times, you can only run ./test --no-setup . This will only reset
the dataset as its initial state and re-run the test. You might have to run
the tests some times before experiencing it... but I'm always able with only
2-3 runs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8583) Check for Thread.start()
[
https://issues.apache.org/jira/browse/CASSANDRA-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286419#comment-14286419
]
Krzysztof Styrc commented on CASSANDRA-8583:
I've made some deeper look at these 6 Thread.start() use cases.
Cases 1), 2) and 4) create threads for short-running tasks, which might cause
some delays in their context (InputStream decompression; SSTable write to disk,
setting timeout 5s to read version from the stream; respectively).
{{org.apache.cassandra.streaming.ConnectionHandler}} creates separate thread
for handling incoming/outgoing messages. One ConnectionHandler is created per
StreamSession (streaming one or more section of one or more sstables to a
specific remote node). It also seems a relatively short-running task to me. Is
that correct? Of course it depends on how many sections of how many sstables.
Names of the cases 5) and 6) suggest long-running operation, so
{{Thread(...).start()}} seems valid.
To sum up:
Cases 1) 2) and 4) are valid for this issue. I'm not sure about case 3).
Check for Thread.start()
Key: CASSANDRA-8583
URL: https://issues.apache.org/jira/browse/CASSANDRA-8583
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Robert Stupp
Priority: Minor
Old classes sometimes still use
{noformat}
new Thread(...).start()
{noformat}
which might be costly.
This ticket's about to find and possibly fix such code.
Locations in code worth to investigate (IMO). This list is not prioritized -
it's just the order I've found Thread.start()
#
{{org.apache.cassandra.streaming.compress.CompressedInputStream#CompressedInputStream}}
creates one thread per input stream to decompress in a separate thread. If
necessary, should be easily replaceable with a thread-pool
#
{{org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter#SSTableSimpleUnsortedWriter(java.io.File,
org.apache.cassandra.config.CFMetaData,
org.apache.cassandra.dht.IPartitioner, long)}} creates one thread per write.
If necessary, should be easily replaceable with a thread-pool
# {{org.apache.cassandra.streaming.ConnectionHandler.MessageHandler#start}}
creates one thread. If necessary, should be easily replaceable with a
thread-pool.
# {{org.apache.cassandra.net.OutboundTcpConnection#handshakeVersion}} creates
one thread just to implement a timeout. Not sure why not just using
{{Socket.setSoTimeout}}
#
{{org.apache.cassandra.service.StorageService#forceRepairAsync(java.lang.String,
org.apache.cassandra.repair.messages.RepairOption)}} creates one thread per
repair. Not sure whether it's worth to investigate this one, since repairs
are long running operations
# {{org.apache.cassandra.db.index.SecondaryIndex#buildIndexAsync}} creates a
thread. Not sure whether it's worth to investigate this one.
Beside these, there are threads used in {{MessagingService}} and for
streaming (blocking I/O model). These could be changed by using non-blocking
I/O - but that's a much bigger task with much higher risks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[
https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishan Karunaratne updated CASSANDRA-8665:
--
Description:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully and then gave up.
was:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Cassandra does not start with NPE in
ColumnFamilyStore.removeUnfinishedCompactionLeftovers
--
Key: CASSANDRA-8665
URL: https://issues.apache.org/jira/browse/CASSANDRA-8665
Project: Cassandra
Issue Type: Bug
Environment: C* 2.1.2 | ruby-driver 1.2
Reporter: Kishan Karunaratne
Fix For: 2.1.3
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 -
Initializing system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
[jira] [Updated] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[
https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishan Karunaratne updated CASSANDRA-8665:
--
Environment: Ubuntu 12.04 | C* 2.1.2 | ruby-driver 1.2 (was: C* 2.1.2 |
ruby-driver 1.2)
Cassandra does not start with NPE in
ColumnFamilyStore.removeUnfinishedCompactionLeftovers
--
Key: CASSANDRA-8665
URL: https://issues.apache.org/jira/browse/CASSANDRA-8665
Project: Cassandra
Issue Type: Bug
Environment: Ubuntu 12.04 | C* 2.1.2 | ruby-driver 1.2
Reporter: Kishan Karunaratne
Fix For: 2.1.3
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:18:25,780
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 -
Initializing system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 -
Initializing system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully (this error occurred
twice) and then gave up; it seems like a corrupt Data.db file? The endurance
test consists of a chaos rhino which randomly rolling restarts a node.
The only other significant error is TombstoneOverwhelmingException and is
probably unrelated:
{noformat}
/mnt/systemlogs/system.log:ERROR [HintedHandoff:2] 2015-01-17 12:46:32,378
SliceQueryFilter.java:218 - Scanned over 10 tombstones in system.hints;
query aborted (see tombstone_failure_threshold)
/mnt/systemlogs/system.log:ERROR [HintedHandoff:2] 2015-01-17 12:46:32,416
CassandraDaemon.java:170 - Exception in thread Thread[HintedHandoff:2,1,main]
/mnt/systemlogs/system.log-org.apache.cassandra.db.filter.TombstoneOverwhelmingException:
null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:220)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:107)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:81)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:69)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:320)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1885)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1693)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:378)
~[main/:na]
/mnt/systemlogs/system.log- at
[jira] [Commented] (CASSANDRA-8358) Bundled tools shouldn't be using Thrift API
[ https://issues.apache.org/jira/browse/CASSANDRA-8358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286608#comment-14286608 ] Philip Thompson commented on CASSANDRA-8358: I am now just waiting on CASSANDRA-8622 to verify I have not broken pig-test, and JAVA-312 for the changes to BulkLoader. The patch is otherwise complete. Bundled tools shouldn't be using Thrift API --- Key: CASSANDRA-8358 URL: https://issues.apache.org/jira/browse/CASSANDRA-8358 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Philip Thompson Fix For: 3.0 In 2.1, we switched cqlsh to the python-driver. In 3.0, we got rid of cassandra-cli. Yet there is still code that's using legacy Thrift API. We want to convert it all to use the java-driver instead. 1. BulkLoader uses Thrift to query the schema tables. It should be using java-driver metadata APIs directly instead. 2. o.a.c.hadoop.cql3.CqlRecordWriter is using Thrift 3. o.a.c.hadoop.ColumnFamilyRecordReader is using Thrift 4. o.a.c.hadoop.AbstractCassandraStorage is using Thrift 5. o.a.c.hadoop.pig.CqlStorage is using Thrift Some of the things listed above use Thrift to get the list of partition key columns or clustering columns. Those should be converted to use the Metadata API of the java-driver. Somewhat related to that, we also have badly ported code from Thrift in o.a.c.hadoop.cql3.CqlRecordReader (see fetchKeys()) that manually fetches columns from schema tables instead of properly using the driver's Metadata API. We need all of it fixed. One exception, for now, is o.a.c.hadoop.AbstractColumnFamilyInputFormat - it's using Thrift for its describe_splits_ex() call that cannot be currently replaced by any java-driver call (?). Once this is done, we can stop starting Thrift RPC port by default in cassandra.yaml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java src/java/org/apache/cassandra/db/ColumnFamilyStore.java test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06cd494c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06cd494c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06cd494c Branch: refs/heads/trunk Commit: 06cd494c1496ec96886ed41ff3207847631986c9 Parents: 0c2eaa9 cc5fb19 Author: Aleksey Yeschenko alek...@apache.org Authored: Thu Jan 22 03:17:55 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Thu Jan 22 03:17:55 2015 +0300 -- CHANGES.txt | 1 + .../cassandra/db/ArrayBackedSortedColumns.java | 96 +-- .../apache/cassandra/db/AtomicBTreeColumns.java | 5 + .../org/apache/cassandra/db/ColumnFamily.java | 6 ++ .../apache/cassandra/db/ColumnFamilyStore.java | 7 +- .../cassandra/utils/BatchRemoveIterator.java| 32 +++ .../db/ArrayBackedSortedColumnsTest.java| 99 +++- 7 files changed, 236 insertions(+), 10 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/06cd494c/CHANGES.txt -- diff --cc CHANGES.txt index 9cd8189,0d08cce..a94ca04 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,70 -1,9 +1,71 @@@ -2.0.13: - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) +2.1.3 + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating error message + for some CQL syntax errors (CASSANDRA-8455) + * Scale memtable slab allocation logarithmically (CASSANDRA-7882) + * cassandra-stress simultaneous inserts over same seed (CASSANDRA-7964) + * Reduce cassandra-stress sampling memory requirements (CASSANDRA-7926) + * Ensure memtable flush cannot expire commit log entries from its future (CASSANDRA-8383) + * Make read defrag async to reclaim memtables (CASSANDRA-8459) + * Remove tmplink files for offline compactions (CASSANDRA-8321) + * Reduce maxHintsInProgress (CASSANDRA-8415) + * BTree updates may call provided update function twice
[1/3] cassandra git commit: Add batch remove iterator to ABSC
Repository: cassandra
Updated Branches:
refs/heads/trunk 39108df7c - 2b4cd105e
Add batch remove iterator to ABSC
patch by Jimmy MÃ¥rdell; reviewed by Richard Low for CASSANDRA-8414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc5fb19e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc5fb19e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc5fb19e
Branch: refs/heads/trunk
Commit: cc5fb19e5c864110b798375f43ec1597904d03ab
Parents: ae380da
Author: Jimmy MÃ¥rdell ya...@spotify.com
Authored: Thu Jan 22 02:16:10 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jan 22 02:16:10 2015 +0300
--
CHANGES.txt | 1 +
.../cassandra/db/ArrayBackedSortedColumns.java | 89 ++-
.../org/apache/cassandra/db/ColumnFamily.java | 32 +++
.../apache/cassandra/db/ColumnFamilyStore.java | 6 +-
.../cassandra/utils/BatchRemoveIterator.java| 32 +++
.../db/ArrayBackedSortedColumnsTest.java| 93
6 files changed, 249 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc5fb19e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6604783..0d08cce 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.0.13:
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
+ * Add batch remove iterator to ABSC (CASSANDRA-8414)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc5fb19e/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 389e0f8..8d553be 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -29,6 +29,7 @@ import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.filter.ColumnSlice;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.utils.Allocator;
+import org.apache.cassandra.utils.BatchRemoveIterator;
/**
* A ColumnFamily backed by an ArrayList.
@@ -54,14 +55,14 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
{
super(metadata);
this.reversed = reversed;
-this.columns = new ArrayListColumn();
+this.columns = new ArrayList();
}
private ArrayBackedSortedColumns(CollectionColumn columns, CFMetaData
metadata, boolean reversed)
{
super(metadata);
this.reversed = reversed;
-this.columns = new ArrayListColumn(columns);
+this.columns = new ArrayList(columns);
}
public ColumnFamily.Factory getFactory()
@@ -292,6 +293,90 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
return new SlicesIterator(columns, getComparator(), slices, !reversed);
}
+@Override
+public BatchRemoveIteratorColumn batchRemoveIterator()
+{
+return new BatchRemoveIteratorColumn()
+{
+private IteratorColumn iter = iterator();
+private BitSet removedIndexes = new BitSet(columns.size());
+private int idx = -1;
+private boolean shouldCallNext = true;
+private boolean isCommitted = false;
+private boolean removedAnything = false;
+
+public void commit()
+{
+if (isCommitted)
+throw new IllegalStateException();
+isCommitted = true;
+
+if (!removedAnything)
+return;
+
+// the lowest index both not visited and known to be not
removed
+int keepIdx = removedIndexes.nextClearBit(0);
+// the running total of kept items
+int resultLength = 0;
+// start from the first not-removed cell, and shift left.
+int removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
+while (removeIdx = 0)
+{
+int length = removeIdx - keepIdx;
+if (length 0)
+{
+copy(keepIdx, resultLength, length);
+resultLength += length;
+}
+keepIdx = removedIndexes.nextClearBit(removeIdx + 1);
+if (keepIdx 0)
+keepIdx = columns.size();
+removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
+}
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk
Conflicts:
src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2b4cd105
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2b4cd105
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2b4cd105
Branch: refs/heads/trunk
Commit: 2b4cd105e8854543e1962888a2cb2fe9fccc4d17
Parents: 39108df 06cd494
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jan 22 03:22:34 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jan 22 03:22:34 2015 +0300
--
CHANGES.txt | 1 +
.../cassandra/db/ArrayBackedSortedColumns.java | 96 +++--
.../apache/cassandra/db/AtomicBTreeColumns.java | 5 +
.../org/apache/cassandra/db/ColumnFamily.java | 6 ++
.../apache/cassandra/db/ColumnFamilyStore.java | 7 +-
.../cassandra/utils/BatchRemoveIterator.java| 32 ++
.../db/ArrayBackedSortedColumnsTest.java| 103 ++-
7 files changed, 238 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b4cd105/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b4cd105/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --cc src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index e5a731c,64752e3..9574d09
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@@ -32,8 -27,8 +27,9 @@@ import org.apache.cassandra.config.CFMe
import org.apache.cassandra.db.composites.CellName;
import org.apache.cassandra.db.composites.Composite;
import org.apache.cassandra.db.filter.ColumnSlice;
+ import org.apache.cassandra.utils.BatchRemoveIterator;
import org.apache.cassandra.utils.memory.AbstractAllocator;
+import org.apache.cassandra.utils.SearchIterator;
/**
* A ColumnFamily backed by an array.
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b4cd105/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b4cd105/src/java/org/apache/cassandra/db/ColumnFamily.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b4cd105/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b4cd105/test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java
--
diff --cc test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java
index 46fe812,18851d4..0fdabe9
--- a/test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java
+++ b/test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java
@@@ -27,33 -25,19 +27,36 @@@ import org.junit.Test
import static org.junit.Assert.*;
+ import com.google.common.collect.Sets;
+
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.config.KSMetaData;
import org.apache.cassandra.config.Schema;
- import org.apache.cassandra.exceptions.ConfigurationException;
- import org.apache.cassandra.locator.SimpleStrategy;
- import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.db.composites.*;
import org.apache.cassandra.db.filter.ColumnSlice;
import org.apache.cassandra.db.marshal.Int32Type;
-import org.apache.cassandra.utils.BatchRemoveIterator;
++import org.apache.cassandra.exceptions.ConfigurationException;
++import org.apache.cassandra.locator.SimpleStrategy;
+ import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.SearchIterator;
++import org.apache.cassandra.utils.BatchRemoveIterator;
-public class ArrayBackedSortedColumnsTest extends SchemaLoader
+public class ArrayBackedSortedColumnsTest
{
+private static final String KEYSPACE1 = ArrayBackedSortedColumnsTest;
+private static final String CF_STANDARD1 = Standard1;
+
+@BeforeClass
+public static void defineSchema() throws ConfigurationException
+{
+SchemaLoader.prepareServer();
+SchemaLoader.createKeyspace(KEYSPACE1,
+SimpleStrategy.class,
+
[jira] [Resolved] (CASSANDRA-8563) cqlsh broken for some thrift created tables.
[ https://issues.apache.org/jira/browse/CASSANDRA-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs resolved CASSANDRA-8563. Resolution: Fixed The bundled python driver has been upgraded in 7dc94abd57e51e7e69aad446922b4b007c29d2d4, which resolves this issue. cqlsh broken for some thrift created tables. Key: CASSANDRA-8563 URL: https://issues.apache.org/jira/browse/CASSANDRA-8563 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Assignee: Tyler Hobbs Labels: cqlsh Fix For: 2.1.3 The new python driver based cqlsh is broken for some tables. This was fixed recently in: https://datastax-oss.atlassian.net/browse/PYTHON-192 So we should pull in a new version of the python driver. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8372) cqlsh - throws TypeError on SELECT of a map with blob keys
[
https://issues.apache.org/jira/browse/CASSANDRA-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs resolved CASSANDRA-8372.
Resolution: Fixed
The bundled python driver has been upgraded in
7dc94abd57e51e7e69aad446922b4b007c29d2d4, which resolves this issue.
cqlsh - throws TypeError on SELECT of a map with blob keys
--
Key: CASSANDRA-8372
URL: https://issues.apache.org/jira/browse/CASSANDRA-8372
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Nathan Cormier
Assignee: Tyler Hobbs
Priority: Minor
Labels: cqlsh
Fix For: 2.1.3
CREATE TABLE users (user_id bigint PRIMARY KEY, config mapblob, int);
INSERT INTO users (user_id, config) VALUES (123, {0xaa: 1});
SELECT user_id FROM users;
This works fine.
SELECT * FROM users;
Any SELECT with the map field will result in this error:
{code}
Traceback (most recent call last):
File /usr/local/bin/cqlsh, line 909, in perform_simple_statement
rows = self.session.execute(statement, trace=self.tracing_enabled)
File /Library/Python/2.7/site-packages/cassandra/cluster.py, line 1281,
in execute
result = future.result(timeout)
File /Library/Python/2.7/site-packages/cassandra/cluster.py, line 2771,
in result
raise self._final_exception
TypeError: unhashable type: 'bytearray'
{code}
This is cqlsh version 5.0.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8651) Add support for running on Apache Mesos
[ https://issues.apache.org/jira/browse/CASSANDRA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286450#comment-14286450 ] Albert P Tobey commented on CASSANDRA-8651: --- The folks at Mesosphere are working on building an executor for Mesos and are hoping to upstream any components that make sense to live in the Cassandra tree. It sounds like there could be a custom MesosSeedProvider. There's also a question of whether or not it makes sense to have the executor code live in the Cassandra tree. I think that will be easier to answer once it exists. For now, I don't think they need anything from the Cassandra developers. This ticket exists to make the work visible to the community. Add support for running on Apache Mesos --- Key: CASSANDRA-8651 URL: https://issues.apache.org/jira/browse/CASSANDRA-8651 Project: Cassandra Issue Type: Task Reporter: Ben Whitehead Priority: Minor Fix For: 3.0 As a user of Apache Mesos I would like to be able to run Cassandra on my Mesos cluster. This would entail integration of Cassandra on Mesos through the creation of a production level Mesos framework. This would enable me to avoid static partitioning and inefficiencies and run Cassandra as part of my data center infrastructure. http://mesos.apache.org/documentation/latest/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8616) sstable2json may result in commit log segments be written
[ https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-8616: --- Reviewer: Tyler Hobbs sstable2json may result in commit log segments be written - Key: CASSANDRA-8616 URL: https://issues.apache.org/jira/browse/CASSANDRA-8616 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Tyler Hobbs Assignee: Yuki Morishita Fix For: 2.0.13 Attachments: 8161-2.0.txt There was a report of sstable2json causing commitlog segments to be written out when run. I haven't attempted to reproduce this yet, so that's all I know for now. Since sstable2json loads the conf and schema, I'm thinking that it may inadvertently be triggering the commitlog code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8616) sstable2json may result in commit log segments be written
[ https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286501#comment-14286501 ] Tyler Hobbs commented on CASSANDRA-8616: +1 sstable2json may result in commit log segments be written - Key: CASSANDRA-8616 URL: https://issues.apache.org/jira/browse/CASSANDRA-8616 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Tyler Hobbs Assignee: Yuki Morishita Fix For: 2.0.13 Attachments: 8161-2.0.txt There was a report of sstable2json causing commitlog segments to be written out when run. I haven't attempted to reproduce this yet, so that's all I know for now. Since sstable2json loads the conf and schema, I'm thinking that it may inadvertently be triggering the commitlog code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Add batch remove iterator to ABSC
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 ae380da9e - cc5fb19e5
Add batch remove iterator to ABSC
patch by Jimmy MÃ¥rdell; reviewed by Richard Low for CASSANDRA-8414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc5fb19e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc5fb19e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc5fb19e
Branch: refs/heads/cassandra-2.0
Commit: cc5fb19e5c864110b798375f43ec1597904d03ab
Parents: ae380da
Author: Jimmy MÃ¥rdell ya...@spotify.com
Authored: Thu Jan 22 02:16:10 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jan 22 02:16:10 2015 +0300
--
CHANGES.txt | 1 +
.../cassandra/db/ArrayBackedSortedColumns.java | 89 ++-
.../org/apache/cassandra/db/ColumnFamily.java | 32 +++
.../apache/cassandra/db/ColumnFamilyStore.java | 6 +-
.../cassandra/utils/BatchRemoveIterator.java| 32 +++
.../db/ArrayBackedSortedColumnsTest.java| 93
6 files changed, 249 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc5fb19e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6604783..0d08cce 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.0.13:
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
+ * Add batch remove iterator to ABSC (CASSANDRA-8414)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc5fb19e/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 389e0f8..8d553be 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -29,6 +29,7 @@ import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.filter.ColumnSlice;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.utils.Allocator;
+import org.apache.cassandra.utils.BatchRemoveIterator;
/**
* A ColumnFamily backed by an ArrayList.
@@ -54,14 +55,14 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
{
super(metadata);
this.reversed = reversed;
-this.columns = new ArrayListColumn();
+this.columns = new ArrayList();
}
private ArrayBackedSortedColumns(CollectionColumn columns, CFMetaData
metadata, boolean reversed)
{
super(metadata);
this.reversed = reversed;
-this.columns = new ArrayListColumn(columns);
+this.columns = new ArrayList(columns);
}
public ColumnFamily.Factory getFactory()
@@ -292,6 +293,90 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
return new SlicesIterator(columns, getComparator(), slices, !reversed);
}
+@Override
+public BatchRemoveIteratorColumn batchRemoveIterator()
+{
+return new BatchRemoveIteratorColumn()
+{
+private IteratorColumn iter = iterator();
+private BitSet removedIndexes = new BitSet(columns.size());
+private int idx = -1;
+private boolean shouldCallNext = true;
+private boolean isCommitted = false;
+private boolean removedAnything = false;
+
+public void commit()
+{
+if (isCommitted)
+throw new IllegalStateException();
+isCommitted = true;
+
+if (!removedAnything)
+return;
+
+// the lowest index both not visited and known to be not
removed
+int keepIdx = removedIndexes.nextClearBit(0);
+// the running total of kept items
+int resultLength = 0;
+// start from the first not-removed cell, and shift left.
+int removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
+while (removeIdx = 0)
+{
+int length = removeIdx - keepIdx;
+if (length 0)
+{
+copy(keepIdx, resultLength, length);
+resultLength += length;
+}
+keepIdx = removedIndexes.nextClearBit(removeIdx + 1);
+if (keepIdx 0)
+keepIdx = columns.size();
+removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
+
[jira] [Created] (CASSANDRA-8664) New cqlsh formatter for cassandra-driver type supporting nested, frozen collections
Adam Holmberg created CASSANDRA-8664: Summary: New cqlsh formatter for cassandra-driver type supporting nested, frozen collections Key: CASSANDRA-8664 URL: https://issues.apache.org/jira/browse/CASSANDRA-8664 Project: Cassandra Issue Type: New Feature Reporter: Adam Holmberg Priority: Minor Fix For: 2.1.3 Attachments: cqlshlib-patch.txt Supporting nested frozen collections required a new type returned for map objects in cassandra-driver. The attached patch adds a formatter for this type. References: [Driver issue|https://datastax-oss.atlassian.net/browse/PYTHON-186] [Changeset|https://github.com/datastax/python-driver/pull/242/files] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8635) STCS cold sstable omission does not handle overwrites without reads
[ https://issues.apache.org/jira/browse/CASSANDRA-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286481#comment-14286481 ] Tyler Hobbs commented on CASSANDRA-8635: This definitely seems like it will help many cases, but it seems like there are still a couple of problematic scenarios: * What if the cold sstables overlap with each other but not with any hot sstables? * What if they all overlap by 75% and thus fall below the threshold? If we go with this strategy, it seems like we still need a safety mechanism to ensure that number of sstables never blows up, such as a max number of sstables. After thinking about this for a while, I'm also tempted to scrap the whole don't-compact-cold-sstables in STCS approach in favor of using DTCS. (This was implemented prior to DTCS being proposed.) Since STCS somewhat randomly mixes data within sstables, the only pattern that is likely to generate cold sstables is when old data is read infrequently, and DTCS addresses many of those patterns more effectively. Are there any opinions on this? STCS cold sstable omission does not handle overwrites without reads --- Key: CASSANDRA-8635 URL: https://issues.apache.org/jira/browse/CASSANDRA-8635 Project: Cassandra Issue Type: Bug Components: Core Reporter: Tyler Hobbs Assignee: Marcus Eriksson Priority: Critical Fix For: 2.1.3 Attachments: 0001-Include-cold-sstables-in-compactions-if-they-overlap.patch In 2.1, STCS may omit cold SSTables from compaction (CASSANDRA-6109). If data is regularly overwritten or deleted (but not enough to trigger a single-sstable tombstone purging compaction), data size on disk may continuously grow if: * The table receives very few reads * The reads only touch the newest SSTables Basically, if the overwritten data is never read and there aren't many tombstones, STCS has no incentive to compact the sstables. We should take sstable overlap into consideration as well as coldness to address this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[
https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishan Karunaratne updated CASSANDRA-8665:
--
Description:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully (this error occurred twice)
and then gave up; it seems like a corrupt Data.db file? The endurance test
consists of a chaos rhino which randomly rolling restarts a node.
was:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully and then gave up; it seems
like a corrupt Data.db file? The endurance test consists of a chaos rhino which
randomly rolling restarts a node.
Cassandra does not start with NPE in
ColumnFamilyStore.removeUnfinishedCompactionLeftovers
--
Key: CASSANDRA-8665
[jira] [Updated] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[
https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishan Karunaratne updated CASSANDRA-8665:
--
Description:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:18:25,780
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully (this error occurred twice)
and then gave up; it seems like a corrupt Data.db file? The endurance test
consists of a chaos rhino which randomly rolling restarts a node.
The only other significant error is TombstoneOverwhelmingException and is
probably unrelated:
{noformat}
/mnt/systemlogs/system.log:ERROR [HintedHandoff:2] 2015-01-17 12:46:32,378
SliceQueryFilter.java:218 - Scanned over 10 tombstones in system.hints;
query aborted (see tombstone_failure_threshold)
/mnt/systemlogs/system.log:ERROR [HintedHandoff:2] 2015-01-17 12:46:32,416
CassandraDaemon.java:170 - Exception in thread Thread[HintedHandoff:2,1,main]
/mnt/systemlogs/system.log-org.apache.cassandra.db.filter.TombstoneOverwhelmingException:
null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:220)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:107)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:81)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:69)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:320)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1885)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1693)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:378)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:337)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.HintedHandOffManager.access$400(HintedHandOffManager.java:88)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.HintedHandOffManager$5.run(HintedHandOffManager.java:548)
~[main/:na]
/mnt/systemlogs/system.log- at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_25]
/mnt/systemlogs/system.log- at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
~[na:1.7.0_25]
/mnt/systemlogs/system.log-
cassandra git commit: Upgrade bundled python driver to badad6d40
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 06cd494c1 - 7dc94abd5 Upgrade bundled python driver to badad6d40 Patch by Tyler Hobbs for CASSANDRA-8563 and CASSANDRA-8372 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7dc94abd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7dc94abd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7dc94abd Branch: refs/heads/cassandra-2.1 Commit: 7dc94abd57e51e7e69aad446922b4b007c29d2d4 Parents: 06cd494 Author: Tyler Hobbs tylerho...@apache.org Authored: Wed Jan 21 18:29:26 2015 -0600 Committer: Tyler Hobbs tylerho...@apache.org Committed: Wed Jan 21 18:30:44 2015 -0600 -- CHANGES.txt | 2 ++ lib/cassandra-driver-internal-only-2.1.3.post.zip | Bin 0 - 138474 bytes lib/cassandra-driver-internal-only-2.1.3.zip | Bin 132360 - 0 bytes 3 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dc94abd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index a94ca04..54f734c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,6 @@ 2.1.3 + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) * Add tooling to detect hot partitions (CASSANDRA-7974) * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dc94abd/lib/cassandra-driver-internal-only-2.1.3.post.zip -- diff --git a/lib/cassandra-driver-internal-only-2.1.3.post.zip b/lib/cassandra-driver-internal-only-2.1.3.post.zip new file mode 100644 index 000..3cedfc4 Binary files /dev/null and b/lib/cassandra-driver-internal-only-2.1.3.post.zip differ http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dc94abd/lib/cassandra-driver-internal-only-2.1.3.zip -- diff --git a/lib/cassandra-driver-internal-only-2.1.3.zip b/lib/cassandra-driver-internal-only-2.1.3.zip deleted file mode 100644 index 7a1502a..000 Binary files a/lib/cassandra-driver-internal-only-2.1.3.zip and /dev/null differ
[jira] [Commented] (CASSANDRA-8514) ArrayIndexOutOfBoundsException in nodetool cfhistograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286442#comment-14286442
]
Jonathan Ellis commented on CASSANDRA-8514:
---
I still think null is a better nothing is here value than empty array (NPE is
more obvious that someone forgot to handle a special case than AIOOB) but I'm
not going to argue, +1 from me.
ArrayIndexOutOfBoundsException in nodetool cfhistograms
---
Key: CASSANDRA-8514
URL: https://issues.apache.org/jira/browse/CASSANDRA-8514
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX
Reporter: Philip Thompson
Assignee: Benjamin Lerer
Fix For: 2.1.3
Attachments: CASSANDRA-8514-V2.txt, cassandra-2.1-8514-1.txt
When running nodetool cfhistograms on 2.1-HEAD, I am seeing the following
exception:
{code}
04:02 PM:~/cstar/cassandra[cassandra-2.1*]$ bin/nodetool cfhistograms
keyspace1 standard1
objc[58738]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java and
/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
error: 0
-- StackTrace --
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.cassandra.utils.EstimatedHistogram.newOffsets(EstimatedHistogram.java:75)
at
org.apache.cassandra.utils.EstimatedHistogram.init(EstimatedHistogram.java:60)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:946)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:250)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:164){code}
I can reproduce this with these simple steps:
Start a new C* 2.1-HEAD node
Run {{cassandra-stress write n=1}}
Run {{nodetool cfhistograms keyspace1 standard1}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[
https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishan Karunaratne updated CASSANDRA-8665:
--
Description:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:18:25,780
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully (this error occurred twice)
and then gave up; it seems like a corrupt Data.db file? The endurance test
consists of a chaos rhino which randomly rolling restarts a node.
was:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully (this error occurred twice)
and then gave up; it seems like a corrupt Data.db file? The endurance test
consists of a chaos rhino which randomly rolling restarts a node.
Cassandra does not start with NPE in
ColumnFamilyStore.removeUnfinishedCompactionLeftovers
--
cassandra git commit: Update cqlsh formatting for upgraded python driver
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 7dc94abd5 - 8092d2c65
Update cqlsh formatting for upgraded python driver
Patch by Adam Holmberg; reviewed by Tyler Hobbs for CASSANDRA-8664
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8092d2c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8092d2c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8092d2c6
Branch: refs/heads/cassandra-2.1
Commit: 8092d2c654bac86265738f4ad8c06440ad2ce4b6
Parents: 7dc94ab
Author: Adam Holmberg adam.holmb...@datastax.com
Authored: Wed Jan 21 18:36:29 2015 -0600
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Wed Jan 21 18:36:29 2015 -0600
--
pylib/cqlshlib/formatting.py | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8092d2c6/pylib/cqlshlib/formatting.py
--
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index 845ffac..f03540d 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -246,6 +246,7 @@ def format_value_map(val, encoding, colormap, time_format,
float_precision, null
displaywidth = 4 * len(subs) + sum(k.displaywidth + v.displaywidth for (k,
v) in subs)
return FormattedValue(bval, coloredval, displaywidth)
formatter_for('OrderedDict')(format_value_map)
+formatter_for('OrderedMap')(format_value_map)
def format_value_utype(val, encoding, colormap, time_format, float_precision,
nullval, **_):
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d3c2c98 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d3c2c98 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d3c2c98 Branch: refs/heads/trunk Commit: 5d3c2c985287ad2f2f899a84ccdcb2d3ce963854 Parents: 0c3720a 8092d2c Author: Tyler Hobbs tylerho...@apache.org Authored: Wed Jan 21 18:37:21 2015 -0600 Committer: Tyler Hobbs tylerho...@apache.org Committed: Wed Jan 21 18:37:21 2015 -0600 -- pylib/cqlshlib/formatting.py | 1 + 1 file changed, 1 insertion(+) --
[1/2] cassandra git commit: Update cqlsh formatting for upgraded python driver
Repository: cassandra
Updated Branches:
refs/heads/trunk 0c3720ab8 - 5d3c2c985
Update cqlsh formatting for upgraded python driver
Patch by Adam Holmberg; reviewed by Tyler Hobbs for CASSANDRA-8664
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8092d2c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8092d2c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8092d2c6
Branch: refs/heads/trunk
Commit: 8092d2c654bac86265738f4ad8c06440ad2ce4b6
Parents: 7dc94ab
Author: Adam Holmberg adam.holmb...@datastax.com
Authored: Wed Jan 21 18:36:29 2015 -0600
Committer: Tyler Hobbs tylerho...@apache.org
Committed: Wed Jan 21 18:36:29 2015 -0600
--
pylib/cqlshlib/formatting.py | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8092d2c6/pylib/cqlshlib/formatting.py
--
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index 845ffac..f03540d 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -246,6 +246,7 @@ def format_value_map(val, encoding, colormap, time_format,
float_precision, null
displaywidth = 4 * len(subs) + sum(k.displaywidth + v.displaywidth for (k,
v) in subs)
return FormattedValue(bval, coloredval, displaywidth)
formatter_for('OrderedDict')(format_value_map)
+formatter_for('OrderedMap')(format_value_map)
def format_value_utype(val, encoding, colormap, time_format, float_precision,
nullval, **_):
[jira] [Resolved] (CASSANDRA-8664) New cqlsh formatter for cassandra-driver type supporting nested, frozen collections
[ https://issues.apache.org/jira/browse/CASSANDRA-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs resolved CASSANDRA-8664. Resolution: Fixed +1, committed as 8092d2c. Thanks! New cqlsh formatter for cassandra-driver type supporting nested, frozen collections --- Key: CASSANDRA-8664 URL: https://issues.apache.org/jira/browse/CASSANDRA-8664 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Adam Holmberg Assignee: Adam Holmberg Priority: Minor Fix For: 2.1.3 Attachments: cqlshlib-patch.txt Supporting nested frozen collections required a new type returned for map objects in cassandra-driver. The attached patch adds a formatter for this type. References: [Driver issue|https://datastax-oss.atlassian.net/browse/PYTHON-186] [Changeset|https://github.com/datastax/python-driver/pull/242/files] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8664) New cqlsh formatter for cassandra-driver type supporting nested, frozen collections
[ https://issues.apache.org/jira/browse/CASSANDRA-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-8664: --- Component/s: Tools New cqlsh formatter for cassandra-driver type supporting nested, frozen collections --- Key: CASSANDRA-8664 URL: https://issues.apache.org/jira/browse/CASSANDRA-8664 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Adam Holmberg Assignee: Adam Holmberg Priority: Minor Fix For: 2.1.3 Attachments: cqlshlib-patch.txt Supporting nested frozen collections required a new type returned for map objects in cassandra-driver. The attached patch adds a formatter for this type. References: [Driver issue|https://datastax-oss.atlassian.net/browse/PYTHON-186] [Changeset|https://github.com/datastax/python-driver/pull/242/files] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8664) New cqlsh formatter for cassandra-driver type supporting nested, frozen collections
[ https://issues.apache.org/jira/browse/CASSANDRA-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-8664: --- Reviewer: Tyler Hobbs Assignee: Adam Holmberg New cqlsh formatter for cassandra-driver type supporting nested, frozen collections --- Key: CASSANDRA-8664 URL: https://issues.apache.org/jira/browse/CASSANDRA-8664 Project: Cassandra Issue Type: New Feature Reporter: Adam Holmberg Assignee: Adam Holmberg Priority: Minor Fix For: 2.1.3 Attachments: cqlshlib-patch.txt Supporting nested frozen collections required a new type returned for map objects in cassandra-driver. The attached patch adds a formatter for this type. References: [Driver issue|https://datastax-oss.atlassian.net/browse/PYTHON-186] [Changeset|https://github.com/datastax/python-driver/pull/242/files] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8622) All of pig-test is failing in trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286529#comment-14286529
]
Brandon Williams commented on CASSANDRA-8622:
-
I added that to lib and got the same original stacktrace.
All of pig-test is failing in trunk
---
Key: CASSANDRA-8622
URL: https://issues.apache.org/jira/browse/CASSANDRA-8622
Project: Cassandra
Issue Type: Test
Components: Hadoop
Reporter: Philip Thompson
Assignee: Brandon Williams
Fix For: 3.0
See http://cassci.datastax.com/job/trunk_pigtest/330/testReport/
Every test in the ant target {{ant pig-test}} has been failing on trunk for a
while now.
{code}
java.lang.ExceptionInInitializerError
at org.apache.log4j.Logger.getLogger(Logger.java:40)
at org.hyperic.sigar.SigarLog.getLogger(SigarLog.java:48)
at org.hyperic.sigar.SigarLog.getLogger(SigarLog.java:44)
at org.hyperic.sigar.SigarLog.debug(SigarLog.java:60)
at org.hyperic.sigar.Sigar.clinit(Sigar.java:108)
at org.apache.cassandra.utils.SigarLibrary.init(SigarLibrary.java:45)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:206)
at
org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:408)
at
org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:52)
at
org.apache.cassandra.pig.PigTestBase.startCassandra(PigTestBase.java:96)
at
org.apache.cassandra.pig.CqlRecordReaderTest.setup(CqlRecordReaderTest.java:63)
at
org.apache.log4j.Log4jLoggerFactory.clinit(Log4jLoggerFactory.java:50)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java src/java/org/apache/cassandra/db/ColumnFamilyStore.java test/unit/org/apache/cassandra/db/ArrayBackedSortedColumnsTest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06cd494c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06cd494c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06cd494c Branch: refs/heads/cassandra-2.1 Commit: 06cd494c1496ec96886ed41ff3207847631986c9 Parents: 0c2eaa9 cc5fb19 Author: Aleksey Yeschenko alek...@apache.org Authored: Thu Jan 22 03:17:55 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Thu Jan 22 03:17:55 2015 +0300 -- CHANGES.txt | 1 + .../cassandra/db/ArrayBackedSortedColumns.java | 96 +-- .../apache/cassandra/db/AtomicBTreeColumns.java | 5 + .../org/apache/cassandra/db/ColumnFamily.java | 6 ++ .../apache/cassandra/db/ColumnFamilyStore.java | 7 +- .../cassandra/utils/BatchRemoveIterator.java| 32 +++ .../db/ArrayBackedSortedColumnsTest.java| 99 +++- 7 files changed, 236 insertions(+), 10 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/06cd494c/CHANGES.txt -- diff --cc CHANGES.txt index 9cd8189,0d08cce..a94ca04 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,70 -1,9 +1,71 @@@ -2.0.13: - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) +2.1.3 + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating error message + for some CQL syntax errors (CASSANDRA-8455) + * Scale memtable slab allocation logarithmically (CASSANDRA-7882) + * cassandra-stress simultaneous inserts over same seed (CASSANDRA-7964) + * Reduce cassandra-stress sampling memory requirements (CASSANDRA-7926) + * Ensure memtable flush cannot expire commit log entries from its future (CASSANDRA-8383) + * Make read defrag async to reclaim memtables (CASSANDRA-8459) + * Remove tmplink files for offline compactions (CASSANDRA-8321) + * Reduce maxHintsInProgress (CASSANDRA-8415) + * BTree updates may call provided update function twice
[1/2] cassandra git commit: Add batch remove iterator to ABSC
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 0c2eaa9cb - 06cd494c1
Add batch remove iterator to ABSC
patch by Jimmy MÃ¥rdell; reviewed by Richard Low for CASSANDRA-8414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc5fb19e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc5fb19e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc5fb19e
Branch: refs/heads/cassandra-2.1
Commit: cc5fb19e5c864110b798375f43ec1597904d03ab
Parents: ae380da
Author: Jimmy MÃ¥rdell ya...@spotify.com
Authored: Thu Jan 22 02:16:10 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jan 22 02:16:10 2015 +0300
--
CHANGES.txt | 1 +
.../cassandra/db/ArrayBackedSortedColumns.java | 89 ++-
.../org/apache/cassandra/db/ColumnFamily.java | 32 +++
.../apache/cassandra/db/ColumnFamilyStore.java | 6 +-
.../cassandra/utils/BatchRemoveIterator.java| 32 +++
.../db/ArrayBackedSortedColumnsTest.java| 93
6 files changed, 249 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc5fb19e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6604783..0d08cce 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.0.13:
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
+ * Add batch remove iterator to ABSC (CASSANDRA-8414)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc5fb19e/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 389e0f8..8d553be 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -29,6 +29,7 @@ import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.filter.ColumnSlice;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.utils.Allocator;
+import org.apache.cassandra.utils.BatchRemoveIterator;
/**
* A ColumnFamily backed by an ArrayList.
@@ -54,14 +55,14 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
{
super(metadata);
this.reversed = reversed;
-this.columns = new ArrayListColumn();
+this.columns = new ArrayList();
}
private ArrayBackedSortedColumns(CollectionColumn columns, CFMetaData
metadata, boolean reversed)
{
super(metadata);
this.reversed = reversed;
-this.columns = new ArrayListColumn(columns);
+this.columns = new ArrayList(columns);
}
public ColumnFamily.Factory getFactory()
@@ -292,6 +293,90 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
return new SlicesIterator(columns, getComparator(), slices, !reversed);
}
+@Override
+public BatchRemoveIteratorColumn batchRemoveIterator()
+{
+return new BatchRemoveIteratorColumn()
+{
+private IteratorColumn iter = iterator();
+private BitSet removedIndexes = new BitSet(columns.size());
+private int idx = -1;
+private boolean shouldCallNext = true;
+private boolean isCommitted = false;
+private boolean removedAnything = false;
+
+public void commit()
+{
+if (isCommitted)
+throw new IllegalStateException();
+isCommitted = true;
+
+if (!removedAnything)
+return;
+
+// the lowest index both not visited and known to be not
removed
+int keepIdx = removedIndexes.nextClearBit(0);
+// the running total of kept items
+int resultLength = 0;
+// start from the first not-removed cell, and shift left.
+int removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
+while (removeIdx = 0)
+{
+int length = removeIdx - keepIdx;
+if (length 0)
+{
+copy(keepIdx, resultLength, length);
+resultLength += length;
+}
+keepIdx = removedIndexes.nextClearBit(removeIdx + 1);
+if (keepIdx 0)
+keepIdx = columns.size();
+removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
+
[jira] [Commented] (CASSANDRA-8414) Avoid loops over array backed iterators that call iter.remove()
[ https://issues.apache.org/jira/browse/CASSANDRA-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286630#comment-14286630 ] Aleksey Yeschenko commented on CASSANDRA-8414: -- All right, committed to both 2.0 and 2.1. I fixed some nits in both - added a missing license header, made everything conform our code style, and made some renames for consistency with the other ABSC iterators we have (esp. in 2.1 impl) Also, in 2.1, made AtomicBTreeColumns#getBatchRemoveIterator() throw UOE instead of adding a dummy implementation to CF. The 2.1 version had three bugs in it that I fixed on commit: 1. maybeSortCells() wasn't being called before returning the iterator 2. at the end of commit, only 'size' was updated, but 'sortedSize' was not 3. unlike the 2.0 version, the 2.1 version was not nullifying the trimmed cells at the end of the array Avoid loops over array backed iterators that call iter.remove() --- Key: CASSANDRA-8414 URL: https://issues.apache.org/jira/browse/CASSANDRA-8414 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Richard Low Assignee: Jimmy Mårdell Labels: performance Fix For: 2.1.3, 2.0.13 Attachments: cassandra-2.0-8414-1.txt, cassandra-2.0-8414-2.txt, cassandra-2.0-8414-3.txt, cassandra-2.0-8414-4.txt, cassandra-2.0-8414-5.txt, cassandra-2.1-8414-5.txt, cassandra-2.1-8414-6.txt I noticed from sampling that sometimes compaction spends almost all of its time in iter.remove() in ColumnFamilyStore.removeDeletedStandard. It turns out that the cf object is using ArrayBackedSortedColumns, so deletes are from an ArrayList. If the majority of your columns are GCable tombstones then this is O(n^2). The data structure should be changed or a copy made to avoid this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
Kishan Karunaratne created CASSANDRA-8665:
-
Summary: Cassandra does not start with NPE in
ColumnFamilyStore.removeUnfinishedCompactionLeftovers
Key: CASSANDRA-8665
URL: https://issues.apache.org/jira/browse/CASSANDRA-8665
Project: Cassandra
Issue Type: Bug
Environment: C* 2.1.2 | ruby-driver 1.2
Reporter: Kishan Karunaratne
Fix For: 2.1.3
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[
https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishan Karunaratne updated CASSANDRA-8665:
--
Description:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully and then gave up; it seems
like a corrupt Data.db file? The endurance test consists of a chaos rhino which
randomly rolling restarts a node.
was:
During a ruby driver endurance/duration test, the following error occurred:
{noformat}
/mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:16:58,044
CassandraDaemon.java:482 - Exception encountered during startup
/mnt/systemlogs/system.log-java.lang.NullPointerException: null
/mnt/systemlogs/system.log- at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
/mnt/systemlogs/system.log- at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
{noformat}
Here is the system.log leading up to the error:
{noformat}
INFO [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing
system.peers
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
INFO [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing
system.local
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
INFO [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 -
Opening
/srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
{noformat}
Cassandra attempted to restart twice unsuccessfully and then gave up.
Cassandra does not start with NPE in
ColumnFamilyStore.removeUnfinishedCompactionLeftovers
--
Key: CASSANDRA-8665
URL: https://issues.apache.org/jira/browse/CASSANDRA-8665
Project: Cassandra
Issue Type: Bug
Environment: C* 2.1.2
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0c3720ab Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0c3720ab Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0c3720ab Branch: refs/heads/trunk Commit: 0c3720ab8003947f1fcb6b2b51feac1d7d134af1 Parents: 2b4cd10 7dc94ab Author: Tyler Hobbs tylerho...@apache.org Authored: Wed Jan 21 18:31:40 2015 -0600 Committer: Tyler Hobbs tylerho...@apache.org Committed: Wed Jan 21 18:31:40 2015 -0600 -- CHANGES.txt | 2 ++ lib/cassandra-driver-internal-only-2.1.3.post.zip | Bin 0 - 138474 bytes lib/cassandra-driver-internal-only-2.1.3.zip | Bin 132360 - 0 bytes 3 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0c3720ab/CHANGES.txt -- diff --cc CHANGES.txt index 2642c23,54f734c..b62282a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,60 -1,7 +1,62 @@@ +3.0 + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Add role based access control (CASSANDRA-7653) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * Improve concurrency of repair (CASSANDRA-6455, 8208) + + 2.1.3 + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) - * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) * Add tooling to detect hot partitions (CASSANDRA-7974) * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267)
[1/2] cassandra git commit: Upgrade bundled python driver to badad6d40
Repository: cassandra Updated Branches: refs/heads/trunk 2b4cd105e - 0c3720ab8 Upgrade bundled python driver to badad6d40 Patch by Tyler Hobbs for CASSANDRA-8563 and CASSANDRA-8372 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7dc94abd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7dc94abd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7dc94abd Branch: refs/heads/trunk Commit: 7dc94abd57e51e7e69aad446922b4b007c29d2d4 Parents: 06cd494 Author: Tyler Hobbs tylerho...@apache.org Authored: Wed Jan 21 18:29:26 2015 -0600 Committer: Tyler Hobbs tylerho...@apache.org Committed: Wed Jan 21 18:30:44 2015 -0600 -- CHANGES.txt | 2 ++ lib/cassandra-driver-internal-only-2.1.3.post.zip | Bin 0 - 138474 bytes lib/cassandra-driver-internal-only-2.1.3.zip | Bin 132360 - 0 bytes 3 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dc94abd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index a94ca04..54f734c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,6 @@ 2.1.3 + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) * Add tooling to detect hot partitions (CASSANDRA-7974) * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dc94abd/lib/cassandra-driver-internal-only-2.1.3.post.zip -- diff --git a/lib/cassandra-driver-internal-only-2.1.3.post.zip b/lib/cassandra-driver-internal-only-2.1.3.post.zip new file mode 100644 index 000..3cedfc4 Binary files /dev/null and b/lib/cassandra-driver-internal-only-2.1.3.post.zip differ http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dc94abd/lib/cassandra-driver-internal-only-2.1.3.zip -- diff --git a/lib/cassandra-driver-internal-only-2.1.3.zip b/lib/cassandra-driver-internal-only-2.1.3.zip deleted file mode 100644 index 7a1502a..000 Binary files a/lib/cassandra-driver-internal-only-2.1.3.zip and /dev/null differ
cassandra git commit: only fetch the bound end type if the slice has an end bound
Repository: cassandra
Updated Branches:
refs/heads/trunk 35783b956 - 9fcb409f5
only fetch the bound end type if the slice has an end bound
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9fcb409f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9fcb409f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9fcb409f
Branch: refs/heads/trunk
Commit: 9fcb409f509c2d24914b9bc6f4124b5c526360bc
Parents: 35783b9
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Thu Jan 22 00:27:40 2015 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Thu Jan 22 00:27:40 2015 -0500
--
src/java/org/apache/cassandra/cql3/restrictions/TokenFilter.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fcb409f/src/java/org/apache/cassandra/cql3/restrictions/TokenFilter.java
--
diff --git a/src/java/org/apache/cassandra/cql3/restrictions/TokenFilter.java
b/src/java/org/apache/cassandra/cql3/restrictions/TokenFilter.java
index 4b5383b..bd04610 100644
--- a/src/java/org/apache/cassandra/cql3/restrictions/TokenFilter.java
+++ b/src/java/org/apache/cassandra/cql3/restrictions/TokenFilter.java
@@ -192,10 +192,10 @@ final class TokenFilter extends
ForwardingPrimaryKeyRestrictions
Token start = deserializeToken(slice.bounds(START,
options).get(0));
BoundType startBoundType = toBoundType(slice.isInclusive(START));
-BoundType endBoundType = toBoundType(slice.isInclusive(END));
if (slice.hasBound(END))
{
+BoundType endBoundType = toBoundType(slice.isInclusive(END));
Token end = deserializeToken(slice.bounds(END,
options).get(0));
if (start.equals(end) (BoundType.OPEN == startBoundType ||
BoundType.OPEN == endBoundType))
[jira] [Commented] (CASSANDRA-8411) Cassandra stress tool fails with NotStrictlyPositiveException on example profiles
[ https://issues.apache.org/jira/browse/CASSANDRA-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286996#comment-14286996 ] Sachin Janani commented on CASSANDRA-8411: -- [~benedict] [~igorm] I tried to reproduce it on latest code but can't find this bug.Is this already fixed? Cassandra stress tool fails with NotStrictlyPositiveException on example profiles - Key: CASSANDRA-8411 URL: https://issues.apache.org/jira/browse/CASSANDRA-8411 Project: Cassandra Issue Type: Bug Components: Tools Environment: Linux Centos Reporter: Igor Meltser Priority: Trivial Trying to run stress tool with provided profile fails: dsc-cassandra-2.1.2/tools $ ./bin/cassandra-stress user n=1 profile=cqlstress-example.yaml ops\(insert=1\) -node INFO 06:21:35 Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) Connected to cluster: Benchmark Cluster INFO 06:21:35 New Cassandra host /:9042 added Datatacenter: datacenter1; Host: /.; Rack: rack1 Datatacenter: datacenter1; Host: /; Rack: rack1 Datatacenter: datacenter1; Host: /; Rack: rack1 INFO 06:21:35 New Cassandra host /:9042 added INFO 06:21:35 New Cassandra host /:9042 added Created schema. Sleeping 3s for propagation. Exception in thread main org.apache.commons.math3.exception.NotStrictlyPositiveException: standard deviation (0) at org.apache.commons.math3.distribution.NormalDistribution.init(NormalDistribution.java:108) at org.apache.cassandra.stress.settings.OptionDistribution$GaussianFactory.get(OptionDistribution.java:418) at org.apache.cassandra.stress.generate.SeedManager.init(SeedManager.java:59) at org.apache.cassandra.stress.settings.SettingsCommandUser.getFactory(SettingsCommandUser.java:78) at org.apache.cassandra.stress.StressAction.run(StressAction.java:61) at org.apache.cassandra.stress.Stress.main(Stress.java:109) The tool is 2.1.2 version, but the tested Cassandra is 2.0.8 version -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8366) Repair grows data on nodes, causes load to become unbalanced
[
https://issues.apache.org/jira/browse/CASSANDRA-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286873#comment-14286873
]
Alan Boudreault commented on CASSANDRA-8366:
[~krummas] I'm currently testing this with the 8316 patch. I am a little bit
worried about the fact that I am not able to get my nodes repaired and
rebalanced properly after a Did not get a positive ... error. This depends
when the issue happens but if it happens soon, the inc repair seems to double
my cluster size. I thought that running a full sequential repair, then cleanup
and re-compact all nodes would fix the balancing issue, but it doesn't. I don't
seem to be able to reproduce that balancing issue without incremental repairs
or if the error Did not get positive... doesn't occur. Should I try anything
else to restore my nodes storage to its initial size prior the failed inc
repair?
Repair grows data on nodes, causes load to become unbalanced
Key: CASSANDRA-8366
URL: https://issues.apache.org/jira/browse/CASSANDRA-8366
Project: Cassandra
Issue Type: Bug
Environment: 4 node cluster
2.1.2 Cassandra
Inserts and reads are done with CQL driver
Reporter: Jan Karlsson
Assignee: Alan Boudreault
Attachments: results-1750_inc_repair.txt,
results-500_1_inc_repairs.txt, results-500_2_inc_repairs.txt,
results-500_full_repair_then_inc_repairs.txt,
results-500_inc_repairs_not_parallel.txt, test.sh
There seems to be something weird going on when repairing data.
I have a program that runs 2 hours which inserts 250 random numbers and reads
250 times per second. It creates 2 keyspaces with SimpleStrategy and RF of 3.
I use size-tiered compaction for my cluster.
After those 2 hours I run a repair and the load of all nodes goes up. If I
run incremental repair the load goes up alot more. I saw the load shoot up 8
times the original size multiple times with incremental repair. (from 2G to
16G)
with node 9 8 7 and 6 the repro procedure looked like this:
(Note that running full repair first is not a requirement to reproduce.)
{noformat}
After 2 hours of 250 reads + 250 writes per second:
UN 9 583.39 MB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 584.01 MB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 583.72 MB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 583.84 MB 256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
Repair -pr -par on all nodes sequentially
UN 9 746.29 MB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 751.02 MB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 748.89 MB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 758.34 MB 256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
repair -inc -par on all nodes sequentially
UN 9 2.41 GB256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 2.53 GB256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 2.6 GB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 2.17 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
after rolling restart
UN 9 1.47 GB256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 1.5 GB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 2.46 GB256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 1.19 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
compact all nodes sequentially
UN 9 989.99 MB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 994.75 MB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 1.46 GB256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 758.82 MB 256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
repair -inc -par on all nodes sequentially
UN 9 1.98 GB256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 2.3 GB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 3.71 GB256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 1.68 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
restart once more
UN 9 2 GB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 2.05 GB256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 4.1 GB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 1.68 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
{noformat}
Is there something im missing or is this strange behavior?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: add Robert as developer
Repository: cassandra Updated Branches: refs/heads/trunk b5721b512 - 35783b956 add Robert as developer Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35783b95 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35783b95 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35783b95 Branch: refs/heads/trunk Commit: 35783b9567aecb834bc639230c4b5554e151face Parents: b5721b5 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Wed Jan 21 22:26:54 2015 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Wed Jan 21 22:26:54 2015 -0500 -- build.xml | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/35783b95/build.xml -- diff --git a/build.xml b/build.xml index 353eefd..b52ca57 100644 --- a/build.xml +++ b/build.xml @@ -401,6 +401,7 @@ developer id=marcuse name=Marcus Eriksson/ developer id=mishail name=Mikhail Stepura/ developer id=pmalik name=Prashant Malik/ + developer id=rstupp name=Robert Stupp/ developer id=scode name=Peter Schuller/ developer id=slebresne name=Sylvain Lebresne/ developer id=tylerhobbs name=Tyler Hobbs/
cassandra git commit: minor javadoc fixes
Repository: cassandra
Updated Branches:
refs/heads/trunk 5d3c2c985 - 222ea9583
minor javadoc fixes
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/222ea958
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/222ea958
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/222ea958
Branch: refs/heads/trunk
Commit: 222ea958352dc191a3684e23e526c2289c24582d
Parents: 5d3c2c9
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Wed Jan 21 21:40:21 2015 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Wed Jan 21 21:40:21 2015 -0500
--
build.xml| 1 -
src/java/org/apache/cassandra/auth/IAuthorizer.java | 4 ++--
.../org/apache/cassandra/cql3/MultiColumnRelation.java | 4
src/java/org/apache/cassandra/cql3/Tuples.java | 4
.../cassandra/cql3/statements/RequestValidations.java| 6 ++
src/java/org/apache/cassandra/db/AbstractNativeCell.java | 4
src/java/org/apache/cassandra/db/AtomicBTreeColumns.java | 3 ++-
.../cassandra/db/compaction/CompactionManager.java | 6 --
src/java/org/apache/cassandra/gms/IFailureDetector.java | 4 ++--
src/java/org/apache/cassandra/gms/VersionedValue.java| 8 ++--
.../cassandra/hadoop/cql3/CqlBulkRecordWriter.java | 1 +
.../apache/cassandra/hadoop/cql3/CqlRecordReader.java| 8 ++--
src/java/org/apache/cassandra/io/sstable/Component.java | 2 ++
.../cassandra/io/util/FastByteArrayOutputStream.java | 2 +-
.../org/apache/cassandra/locator/CloudstackSnitch.java | 2 ++
.../cassandra/locator/NetworkTopologyStrategy.java | 6 --
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 3 ++-
src/java/org/apache/cassandra/service/StorageProxy.java | 7 +--
src/java/org/apache/cassandra/tools/SSTableExport.java | 1 -
src/java/org/apache/cassandra/tools/SSTableImport.java | 1 -
.../apache/cassandra/tools/SSTableRepairedAtSetter.java | 3 ++-
.../org/apache/cassandra/utils/concurrent/OpOrder.java | 2 ++
src/java/org/apache/cassandra/utils/obs/OpenBitSet.java | 11 ---
23 files changed, 65 insertions(+), 28 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/222ea958/build.xml
--
diff --git a/build.xml b/build.xml
index fccd009..353eefd 100644
--- a/build.xml
+++ b/build.xml
@@ -447,7 +447,6 @@
artifactId=cassandra-parent
version=${version}/
!-- do NOT remove this, it breaks pig-test --
-dependency groupId=org.slf4j artifactId=slf4j-log4j12
version=1.7.2/
dependency groupId=joda-time artifactId=joda-time version=2.3 /
/artifact:pom
http://git-wip-us.apache.org/repos/asf/cassandra/blob/222ea958/src/java/org/apache/cassandra/auth/IAuthorizer.java
--
diff --git a/src/java/org/apache/cassandra/auth/IAuthorizer.java
b/src/java/org/apache/cassandra/auth/IAuthorizer.java
index 7e3240a..0aaebcf 100644
--- a/src/java/org/apache/cassandra/auth/IAuthorizer.java
+++ b/src/java/org/apache/cassandra/auth/IAuthorizer.java
@@ -48,8 +48,8 @@ public interface IAuthorizer
*
* @param performer User who grants the permissions.
* @param permissions Set of permissions to grant.
- * @param to Name of the role to which the permissions are to be granted.
* @param resource Resource on which to grant the permissions.
+ * @param grantee name of the role to which the permissions are to be
granted.
*
* @throws RequestValidationException
* @throws RequestExecutionException
@@ -80,7 +80,7 @@ public interface IAuthorizer
*matching ones.
* @param resource The resource on which permissions are requested. Can be
null, in which case permissions on all
* resources should be returned.
- * @param of The name of the role whose permissions are requested. Can be
null, in which case permissions of every
+ * @param grantee The name of the role whose permissions are requested.
Can be null, in which case permissions of every
* role should be returned.
*
* @return All of the matching permission that the requesting user is
authorized to know about.
http://git-wip-us.apache.org/repos/asf/cassandra/blob/222ea958/src/java/org/apache/cassandra/cql3/MultiColumnRelation.java
--
diff --git a/src/java/org/apache/cassandra/cql3/MultiColumnRelation.java
b/src/java/org/apache/cassandra/cql3/MultiColumnRelation.java
index d754968..7e7a901 100644
---
[jira] [Created] (CASSANDRA-8666) Simplify logic of ABSC#BatchRemoveIterator#commit()
Aleksey Yeschenko created CASSANDRA-8666: Summary: Simplify logic of ABSC#BatchRemoveIterator#commit() Key: CASSANDRA-8666 URL: https://issues.apache.org/jira/browse/CASSANDRA-8666 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Priority: Trivial Fix For: 2.1.3, 2.0.13 Replace current logic with a straightforward single loop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8563) cqlsh broken for some thrift created tables.
[ https://issues.apache.org/jira/browse/CASSANDRA-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286755#comment-14286755 ] Aleksey Yeschenko commented on CASSANDRA-8563: -- https://datastax-oss.atlassian.net/browse/PYTHON-213 is another one (still unresolved), that deals w/ DCT fully. cqlsh broken for some thrift created tables. Key: CASSANDRA-8563 URL: https://issues.apache.org/jira/browse/CASSANDRA-8563 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Assignee: Tyler Hobbs Labels: cqlsh Fix For: 2.1.3 The new python driver based cqlsh is broken for some tables. This was fixed recently in: https://datastax-oss.atlassian.net/browse/PYTHON-192 So we should pull in a new version of the python driver. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: fix ohc license version
Repository: cassandra Updated Branches: refs/heads/trunk 222ea9583 - b5721b512 fix ohc license version Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5721b51 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5721b51 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5721b51 Branch: refs/heads/trunk Commit: b5721b5120d07895a7ece90064bb7d68004c89b7 Parents: 222ea95 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Wed Jan 21 22:07:56 2015 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Wed Jan 21 22:07:56 2015 -0500 -- lib/licenses/ohc-0.2.1.txt | 201 lib/licenses/ohc-0.3.1.txt | 201 2 files changed, 201 insertions(+), 201 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5721b51/lib/licenses/ohc-0.2.1.txt -- diff --git a/lib/licenses/ohc-0.2.1.txt b/lib/licenses/ohc-0.2.1.txt deleted file mode 100644 index eb6b5d3..000 --- a/lib/licenses/ohc-0.2.1.txt +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 -http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - License shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - Licensor shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - Legal Entity shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - control means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - You (or Your) shall mean an individual or Legal Entity - exercising permissions granted by this License. - - Source form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - Object form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - Work shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - Derivative Works shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - Contribution shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, submitted - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as Not a Contribution. - - Contributor shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor
[jira] [Updated] (CASSANDRA-8666) Simplify logic of ABSC#BatchRemoveIterator#commit()
[ https://issues.apache.org/jira/browse/CASSANDRA-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8666: - Attachment: 8666.txt Simplify logic of ABSC#BatchRemoveIterator#commit() --- Key: CASSANDRA-8666 URL: https://issues.apache.org/jira/browse/CASSANDRA-8666 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Priority: Trivial Fix For: 2.1.3, 2.0.13 Attachments: 8666.txt Replace current logic with a straightforward single loop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8366) Repair grows data on nodes, causes load to become unbalanced
[
https://issues.apache.org/jira/browse/CASSANDRA-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287048#comment-14287048
]
Marcus Eriksson commented on CASSANDRA-8366:
Does it stream data?
Is disk space usage on disk the same as the one in nodetool status?
What happens if you restart the nodes?
What happens if you run a major compaction?
Are they really supposed to get back to the size before the repair? If some
repair sessions actually finish, maybe the data should be there?
Repair grows data on nodes, causes load to become unbalanced
Key: CASSANDRA-8366
URL: https://issues.apache.org/jira/browse/CASSANDRA-8366
Project: Cassandra
Issue Type: Bug
Environment: 4 node cluster
2.1.2 Cassandra
Inserts and reads are done with CQL driver
Reporter: Jan Karlsson
Assignee: Alan Boudreault
Attachments: results-1750_inc_repair.txt,
results-500_1_inc_repairs.txt, results-500_2_inc_repairs.txt,
results-500_full_repair_then_inc_repairs.txt,
results-500_inc_repairs_not_parallel.txt, test.sh
There seems to be something weird going on when repairing data.
I have a program that runs 2 hours which inserts 250 random numbers and reads
250 times per second. It creates 2 keyspaces with SimpleStrategy and RF of 3.
I use size-tiered compaction for my cluster.
After those 2 hours I run a repair and the load of all nodes goes up. If I
run incremental repair the load goes up alot more. I saw the load shoot up 8
times the original size multiple times with incremental repair. (from 2G to
16G)
with node 9 8 7 and 6 the repro procedure looked like this:
(Note that running full repair first is not a requirement to reproduce.)
{noformat}
After 2 hours of 250 reads + 250 writes per second:
UN 9 583.39 MB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 584.01 MB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 583.72 MB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 583.84 MB 256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
Repair -pr -par on all nodes sequentially
UN 9 746.29 MB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 751.02 MB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 748.89 MB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 758.34 MB 256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
repair -inc -par on all nodes sequentially
UN 9 2.41 GB256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 2.53 GB256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 2.6 GB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 2.17 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
after rolling restart
UN 9 1.47 GB256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 1.5 GB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 2.46 GB256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 1.19 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
compact all nodes sequentially
UN 9 989.99 MB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 994.75 MB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 1.46 GB256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 758.82 MB 256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
repair -inc -par on all nodes sequentially
UN 9 1.98 GB256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 2.3 GB 256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 3.71 GB256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 1.68 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
restart once more
UN 9 2 GB 256 ? 28220962-26ae-4eeb-8027-99f96e377406 rack1
UN 8 2.05 GB256 ? f2de6ea1-de88-4056-8fde-42f9c476a090 rack1
UN 7 4.1 GB 256 ? 2b6b5d66-13c8-43d8-855c-290c0f3c3a0b rack1
UN 6 1.68 GB256 ? b8bd67f1-a816-46ff-b4a4-136ad5af6d4b rack1
{noformat}
Is there something im missing or is this strange behavior?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8619) using CQLSSTableWriter gives ConcurrentModificationException
[
https://issues.apache.org/jira/browse/CASSANDRA-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286295#comment-14286295
]
Sylvain Lebresne commented on CASSANDRA-8619:
-
patch lgtm, +1 (I'm not sure we need the {{columnFamily = null}} line but it
doesn't hurt either).
bq. it looks to me like we may also be counting additions twice in the
BufferedWriter
I don't think that's the case but I would agree that it's hard to follow
(though the comment on BufferedWriter tries to explain it). Basically, while
BufferedWriter extends SSUW, it bypass some of its method and in particular the
{{addColumn}} which does the counting for SSUW. So counting is only done by the
call in BufferedWriter. And further note that the {{addColumn}} in BufferWriter
doesn't override the one of SSUW. I can agree that all this is not the cleanest
code ever, but it was a simple way to reuse the code of SSUW. I suspect that in
the future we might just want to deprecate and finally remove SSUW (in favor of
simply CQLSSTableWriter), at which point we can clean that up.
using CQLSSTableWriter gives ConcurrentModificationException
Key: CASSANDRA-8619
URL: https://issues.apache.org/jira/browse/CASSANDRA-8619
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: sun jdk 7
linux - ubuntu
Reporter: Igor Berman
Assignee: Benedict
Fix For: 2.1.3, 2.0.13
Attachments: 8619, TimeSeriesCassandraLoaderTest.java
Using CQLSSTableWriter gives ConcurrentModificationException
I'm trying to load many timeseries into cassandra 2.0.11-1
using 'org.apache.cassandra:cassandra-all:2.0.11'
{noformat}
java.util.ConcurrentModificationException
at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1115)
at java.util.TreeMap$ValueIterator.next(TreeMap.java:1160)
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:126)
at
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:202)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:187)
at
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215)
schema
CREATE TABLE test.sample (ts_id bigint, yr int, t timestamp, v double, tgs
setvarchar, PRIMARY KEY((ts_id,yr), t)) WITH CLUSTERING ORDER BY (t DESC)
AND COMPRESSION = {'sstable_compression': 'LZ4Compressor'};
statement:
INSERT INTO test.sample(ts_id, yr, t, v) VALUES (?,?,?,?)
{noformat}
with .withBufferSizeInMB(128); it happens more than with
.withBufferSizeInMB(256);
code based on
http://planetcassandra.org/blog/using-the-cassandra-bulk-loader-updated/
writer.addRow(tsId, year, new Date(time), value);
Any suggestions will be highly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log
[ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285476#comment-14285476 ] Benedict commented on CASSANDRA-6809: - bq. assuming that the sync period is sane (e.g. ~100ms) The sync period is, by default, 10s, and to my knowledge this is what many users run with - so in general we will only compress each individual segment. This is still sane, since the cluster has redundancy, although a sync period of between 100ms and 500ms might be more suitable for high traffic nodes. Still, it's probably not a big deal since we only care about compression when under saturation, which should mean many segments. I only mention it, since it is an easy extension. This extension also means the sync thread may have compressed data _waiting_ for it when it runs, reducing the latency until sync completion. bq. Let me try to rephrase what you are saying to make sure I understand it correctly: Almost: * single sync thread forms sections at regular time intervals and sends them to compression executor/phase (SPMC queue), * _sync thread waits on futures and syncs each in order_ Or, with the extension: * mutators periodically submit segment to compressor * once compressor completes an entire segment, requestExtraSync() is called (instead of in advanceAllocatingFrom()) bq. Why is this simpler, or of comparable complexity? We have two steps in explanation, instead of five. More importantly, there is no interleaving of events to reason about between the sync threads, and the lastSync is accurate (which is important since this could artificially pause writes). This also means future improvements here are easier and safer to deliver, because we don't have to reason about how they interplay with each other. In particular, marking lastSync roll over after each segment is synced is a natural improvement (to ensure write latencies don't spike under load) but is challenging to introduce with multiple sync threads. Since we don't expect this feature to be used widely (we expect multiple CL disks to be used instead, if you're bottlenecking) the simpler approach seems more sensible to me. bq. Wouldn't the two extra queues waste resources and increase latency? We have zero in the typical case, and one extra queue in the uncommon use case. If we introduce enough threads that compression is faster than disk, then there will be near zero synchronization costs; of course, if that is not the case, and we are bottlenecking on compression still, then we aren't really losing much (a few micros. every few hundred millis, at 250MB/s compression speed), so it doesn't seem likely to be significant. We're now no longer honouring the sync interval; we are syncing more frequently, which may reduce disk throughput. The exact time of syncing in relation to each other may also vary, likely into lock-step under saturation, so that there may be short periods of many competing syncs potentially yielding pathological disk behaviour, and introducing competition for the synchronized blocks inside the segments, in effect introducing a MPMC queue, eliminating those few micros of benefit. (FTR, the MPMC, SPMC, MPSC aspects are likely not important here. The only concern is thread signalling, but this is the wrong order of magnitude to matter when bottlenecking on disk or compression of large chunks) Compressed Commit Log - Key: CASSANDRA-6809 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Branimir Lambov Priority: Minor Labels: performance Fix For: 3.0 Attachments: ComitLogStress.java, logtest.txt It seems an unnecessary oversight that we don't compress the commit log. Doing so should improve throughput, but some care will need to be taken to ensure we use as much of a segment as possible. I propose decoupling the writing of the records from the segments. Basically write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the compressed chunks into a CLS as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6706) Duplicate rows returned when in clause has repeated values
[
https://issues.apache.org/jira/browse/CASSANDRA-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-6706:
--
Attachment: CASSANDRA-6706-trunk.txt
CASSANDRA-6706-2.1.txt
The patch for 2.1 make sure that Cassandra will log a warning the first time a
user execute a query containing an IN restriction with duplicate values on the
primary key.
The patch for trunk add unit tests to check the behavior and fix the
{{CHANGES.txt}} files.
Duplicate rows returned when in clause has repeated values
--
Key: CASSANDRA-6706
URL: https://issues.apache.org/jira/browse/CASSANDRA-6706
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Found on
[cqlsh 4.1.0 | Cassandra 2.0.3-SNAPSHOT | CQL spec 3.0.0 | Thrift protocol
19.38.0]
Reporter: Gavin Casey
Assignee: Benjamin Lerer
Labels: cql
Fix For: 3.0
Attachments: CASSANDRA-6706-2.1.txt, CASSANDRA-6706-trunk.txt
If a value is repeated within an IN clause then repeated rows are returned
for the repeats:
cqlsh create table t1(c1 text primary key);
cqlsh insert into t1(c1) values ('A');
cqlsh select * from t1;
c1
A
cqlsh select * from t1 where c1 = 'A';
c1
A
cqlsh select * from t1 where c1 in( 'A');
c1
A
cqlsh:dslog select * from t1 where c1 in( 'A','A');
c1
A
A
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
