date:20150430

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6770281a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6770281a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6770281a

Branch: refs/heads/cassandra-2.1
Commit: 6770281af00e73b7cc71115f97663918de4d998e
Parents: 54ea2f2 75e858a
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 13:06:31 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 13:06:31 2015 +0200

--
 CHANGES.txt  | 1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 3 +++
 .../db/compaction/DateTieredCompactionStrategy.java  | 8 
 3 files changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6770281a/CHANGES.txt
--
diff --cc CHANGES.txt
index 39bcc7c,8a3e5a9..481fb80
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,95 -1,9 +1,96 @@@
 -2.0.15:
 +2.1.6
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 +Merged from 2.0:
+  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
 - * Do more agressive ttl expiration checks to be able to
 -   drop more sstables (CASSANDRA-8243)
   * IncomingTcpConnection thread is not named (CASSANDRA-9262)
   * Close incoming connections when MessagingService is stopped 
(CASSANDRA-9238)
 +
 +2.1.5
 + * Re-add deprecated cold_reads_to_omit param for backwards compat 
(CASSANDRA-9203)
 + * Make anticompaction visible in compactionstats (CASSANDRA-9098)
 + * Improve nodetool getendpoints documentation about the partition
 +   key parameter (CASSANDRA-6458)
 + * Don't check other keyspaces for schema changes when an user-defined
 +   type is altered (CASSANDRA-9187)
 + * Allow takeColumnFamilySnapshot to take a list of tables (CASSANDRA-8348)
 + * Limit major sstable operations to their canonical representation 
(CASSANDRA-8669)
 + * cqlsh: Add tests for INSERT and UPDATE tab completion (CASSANDRA-9125)
 + * cqlsh: quote column names when needed in COPY FROM inserts (CASSANDRA-9080)
 + * Add generate-idea-files target to build.xml (CASSANDRA-9123)
 + * Do not load read meter for offline operations (CASSANDRA-9082)
 + * cqlsh: Make CompositeType data readable (CASSANDRA-8919)
 + * cqlsh: Fix display of triggers (CASSANDRA-9081)
 + * Fix NullPointerException when deleting or setting an element by index on
 +   a null list collection (CASSANDRA-9077)
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Use long for key count in cfstats

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0656924f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0656924f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0656924f

Branch: refs/heads/trunk
Commit: 0656924fc5feb736f219fdf2dba2138a7a209dcd
Parents: f85df37 6770281
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 13:06:39 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 13:06:39 2015 +0200

--
 CHANGES.txt  | 1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 3 +++
 .../db/compaction/DateTieredCompactionStrategy.java  | 8 
 3 files changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0656924f/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0656924f/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0656924f/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--

cassandra git commit: Disable single sstable tombstone compactions for DTCS by default

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 13af59782 - 75e858a66


Disable single sstable tombstone compactions for DTCS by default

Patch by marcuse; reviewed by carlyeks for CASSANDRA-9234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/75e858a6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/75e858a6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/75e858a6

Branch: refs/heads/cassandra-2.0
Commit: 75e858a66de48a7579fa396ea2e46326c7155f29
Parents: 13af597
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Apr 24 09:35:58 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 13:03:14 2015 +0200

--
 CHANGES.txt  | 1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 3 +++
 .../db/compaction/DateTieredCompactionStrategy.java  | 8 
 3 files changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 22a9515..8a3e5a9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to
drop more sstables (CASSANDRA-8243)
  * IncomingTcpConnection thread is not named (CASSANDRA-9262)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index dc7e43a..30081ca 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -64,6 +64,7 @@ public abstract class AbstractCompactionStrategy
 protected float tombstoneThreshold;
 protected long tombstoneCompactionInterval;
 protected boolean uncheckedTombstoneCompaction;
+protected boolean disableTombstoneCompactions = false;
 
 /**
  * pause/resume/getNextBackgroundTask must synchronize.  This guarantees 
that after pause completes,
@@ -288,6 +289,8 @@ public abstract class AbstractCompactionStrategy
  */
 protected boolean worthDroppingTombstones(SSTableReader sstable, int 
gcBefore)
 {
+if (disableTombstoneCompactions)
+return false;
 // since we use estimations to calculate, there is a chance that 
compaction will not drop tombstones actually.
 // if that happens we will end up in infinite compaction loop, so 
first we check enough if enough time has
 // elapsed since SSTable created.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index cfa9c8a..df28bd4 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -43,6 +43,14 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 super(cfs, options);
 this.estimatedRemainingTasks = 0;
 this.options = new DateTieredCompactionStrategyOptions(options);
+if 
(!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
  !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
+{
+disableTombstoneCompactions = true;
+logger.debug(Disabling tombstone compactions for DTCS);
+}
+else
+logger.debug(Enabling tombstone compactions for DTCS);
+
 }
 
 @Override

[1/2] cassandra git commit: Disable single sstable tombstone compactions for DTCS by default

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 54ea2f244 - 6770281af


Disable single sstable tombstone compactions for DTCS by default

Patch by marcuse; reviewed by carlyeks for CASSANDRA-9234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/75e858a6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/75e858a6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/75e858a6

Branch: refs/heads/cassandra-2.1
Commit: 75e858a66de48a7579fa396ea2e46326c7155f29
Parents: 13af597
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Apr 24 09:35:58 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 13:03:14 2015 +0200

--
 CHANGES.txt  | 1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 3 +++
 .../db/compaction/DateTieredCompactionStrategy.java  | 8 
 3 files changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 22a9515..8a3e5a9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to
drop more sstables (CASSANDRA-8243)
  * IncomingTcpConnection thread is not named (CASSANDRA-9262)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index dc7e43a..30081ca 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -64,6 +64,7 @@ public abstract class AbstractCompactionStrategy
 protected float tombstoneThreshold;
 protected long tombstoneCompactionInterval;
 protected boolean uncheckedTombstoneCompaction;
+protected boolean disableTombstoneCompactions = false;
 
 /**
  * pause/resume/getNextBackgroundTask must synchronize.  This guarantees 
that after pause completes,
@@ -288,6 +289,8 @@ public abstract class AbstractCompactionStrategy
  */
 protected boolean worthDroppingTombstones(SSTableReader sstable, int 
gcBefore)
 {
+if (disableTombstoneCompactions)
+return false;
 // since we use estimations to calculate, there is a chance that 
compaction will not drop tombstones actually.
 // if that happens we will end up in infinite compaction loop, so 
first we check enough if enough time has
 // elapsed since SSTable created.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index cfa9c8a..df28bd4 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -43,6 +43,14 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 super(cfs, options);
 this.estimatedRemainingTasks = 0;
 this.options = new DateTieredCompactionStrategyOptions(options);
+if 
(!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
  !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
+{
+disableTombstoneCompactions = true;
+logger.debug(Disabling tombstone compactions for DTCS);
+}
+else
+logger.debug(Enabling tombstone compactions for DTCS);
+
 }
 
 @Override

[1/3] cassandra git commit: Disable single sstable tombstone compactions for DTCS by default

Repository: cassandra
Updated Branches:
  refs/heads/trunk f85df3741 - 0656924fc


Disable single sstable tombstone compactions for DTCS by default

Patch by marcuse; reviewed by carlyeks for CASSANDRA-9234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/75e858a6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/75e858a6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/75e858a6

Branch: refs/heads/trunk
Commit: 75e858a66de48a7579fa396ea2e46326c7155f29
Parents: 13af597
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Apr 24 09:35:58 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 13:03:14 2015 +0200

--
 CHANGES.txt  | 1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 3 +++
 .../db/compaction/DateTieredCompactionStrategy.java  | 8 
 3 files changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 22a9515..8a3e5a9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to
drop more sstables (CASSANDRA-8243)
  * IncomingTcpConnection thread is not named (CASSANDRA-9262)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index dc7e43a..30081ca 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -64,6 +64,7 @@ public abstract class AbstractCompactionStrategy
 protected float tombstoneThreshold;
 protected long tombstoneCompactionInterval;
 protected boolean uncheckedTombstoneCompaction;
+protected boolean disableTombstoneCompactions = false;
 
 /**
  * pause/resume/getNextBackgroundTask must synchronize.  This guarantees 
that after pause completes,
@@ -288,6 +289,8 @@ public abstract class AbstractCompactionStrategy
  */
 protected boolean worthDroppingTombstones(SSTableReader sstable, int 
gcBefore)
 {
+if (disableTombstoneCompactions)
+return false;
 // since we use estimations to calculate, there is a chance that 
compaction will not drop tombstones actually.
 // if that happens we will end up in infinite compaction loop, so 
first we check enough if enough time has
 // elapsed since SSTable created.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/75e858a6/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index cfa9c8a..df28bd4 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -43,6 +43,14 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 super(cfs, options);
 this.estimatedRemainingTasks = 0;
 this.options = new DateTieredCompactionStrategyOptions(options);
+if 
(!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
  !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
+{
+disableTombstoneCompactions = true;
+logger.debug(Disabling tombstone compactions for DTCS);
+}
+else
+logger.debug(Enabling tombstone compactions for DTCS);
+
 }
 
 @Override

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6770281a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6770281a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6770281a

Branch: refs/heads/trunk
Commit: 6770281af00e73b7cc71115f97663918de4d998e
Parents: 54ea2f2 75e858a
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 13:06:31 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 13:06:31 2015 +0200

--
 CHANGES.txt  | 1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 3 +++
 .../db/compaction/DateTieredCompactionStrategy.java  | 8 
 3 files changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6770281a/CHANGES.txt
--
diff --cc CHANGES.txt
index 39bcc7c,8a3e5a9..481fb80
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,95 -1,9 +1,96 @@@
 -2.0.15:
 +2.1.6
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 +Merged from 2.0:
+  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
 - * Do more agressive ttl expiration checks to be able to
 -   drop more sstables (CASSANDRA-8243)
   * IncomingTcpConnection thread is not named (CASSANDRA-9262)
   * Close incoming connections when MessagingService is stopped 
(CASSANDRA-9238)
 +
 +2.1.5
 + * Re-add deprecated cold_reads_to_omit param for backwards compat 
(CASSANDRA-9203)
 + * Make anticompaction visible in compactionstats (CASSANDRA-9098)
 + * Improve nodetool getendpoints documentation about the partition
 +   key parameter (CASSANDRA-6458)
 + * Don't check other keyspaces for schema changes when an user-defined
 +   type is altered (CASSANDRA-9187)
 + * Allow takeColumnFamilySnapshot to take a list of tables (CASSANDRA-8348)
 + * Limit major sstable operations to their canonical representation 
(CASSANDRA-8669)
 + * cqlsh: Add tests for INSERT and UPDATE tab completion (CASSANDRA-9125)
 + * cqlsh: quote column names when needed in COPY FROM inserts (CASSANDRA-9080)
 + * Add generate-idea-files target to build.xml (CASSANDRA-9123)
 + * Do not load read meter for offline operations (CASSANDRA-9082)
 + * cqlsh: Make CompositeType data readable (CASSANDRA-8919)
 + * cqlsh: Fix display of triggers (CASSANDRA-9081)
 + * Fix NullPointerException when deleting or setting an element by index on
 +   a null list collection (CASSANDRA-9077)
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Use long for key count in cfstats

[jira] [Updated] (CASSANDRA-9151) Anti-compaction is blocking ANTI_ENTROPY stage


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-9151:
--
Attachment: 0001-Do-not-block-stage-when-performing-anticompaction.patch

Attaching patch for fix against cassandra-2.1 branch.

 Anti-compaction is blocking ANTI_ENTROPY stage 
 ---

 Key: CASSANDRA-9151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9151
 Project: Cassandra
  Issue Type: Bug
Reporter: sankalp kohli
Assignee: Yuki Morishita
 Fix For: 3.x, 2.1.x

 Attachments: 
 0001-Do-not-block-stage-when-performing-anticompaction.patch


 In RepairMessageVerbHandler.doVerb case ANTICOMPACTION_REQUEST
 We block on ANTI_ENTROPY stage. This is not good as no other message could be 
 processed till any anticompaction is running. This can cause snapshots for 
 new repairs to fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9104) Unit test failures, trunk + Windows

2015-04-30 Thread Branimir Lambov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521283#comment-14521283
 ] 

Branimir Lambov commented on CASSANDRA-9104:


+1, this is a good solution.

 Unit test failures, trunk + Windows
 ---

 Key: CASSANDRA-9104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9104
 Project: Cassandra
  Issue Type: Test
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
  Labels: Windows
 Fix For: 3.x

 Attachments: 9104_CFSTest.txt, 9104_KeyCache.txt, 
 9104_KeyCache_ScrubTest_v2.txt, 9104_RecoveryManager.txt, 
 9104_RecoveryManager_v2.txt, 9104_ScrubTest.txt


 Variety of different test failures have cropped up over the past 2-3 weeks:
 h6. -org.apache.cassandra.cql3.UFTest FAILED (timeout)- // No longer failing 
 / timing out
 h6. 
 testLoadNewSSTablesAvoidsOverwrites(org.apache.cassandra.db.ColumnFamilyStoreTest):
FAILED
 {noformat}
12 SSTables unexpectedly exist
junit.framework.AssertionFailedError: 12 SSTables unexpectedly exist
at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testLoadNewSSTablesAvoidsOverwrites(ColumnFamilyStoreTest.java:1896)
 {noformat}
 h6. org.apache.cassandra.db.KeyCacheTest FAILED
 {noformat}
expected:4 but was:2
junit.framework.AssertionFailedError: expected:4 but was:2
at 
 org.apache.cassandra.db.KeyCacheTest.assertKeyCacheSize(KeyCacheTest.java:221)
at org.apache.cassandra.db.KeyCacheTest.testKeyCache(KeyCacheTest.java:181)
 {noformat}
 h6. RecoveryManagerTest:
 {noformat}
org.apache.cassandra.db.RecoveryManagerTest FAILED
org.apache.cassandra.db.RecoveryManager2Test FAILED
org.apache.cassandra.db.RecoveryManager3Test FAILED
org.apache.cassandra.db.RecoveryManagerTruncateTest FAILED
All are the following:
   java.nio.file.AccessDeniedException: 
 build\test\cassandra\commitlog;0\CommitLog-5-1427995105229.log
   FSWriteError in 
 build\test\cassandra\commitlog;0\CommitLog-5-1427995105229.log
  at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:128)
  at 
 org.apache.cassandra.db.commitlog.CommitLogSegmentManager.recycleSegment(CommitLogSegmentManager.java:360)
  at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:156)
  at 
 org.apache.cassandra.db.RecoveryManagerTest.testNothingToRecover(RecoveryManagerTest.java:75)
   Caused by: java.nio.file.AccessDeniedException: 
 build\test\cassandra\commitlog;0\CommitLog-5-1427995105229.log
  at 
 sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
  at 
 sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
  at 
 sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
  at 
 sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
  at 
 sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
  at java.nio.file.Files.delete(Files.java:1079)
  at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:124)
 {noformat}
 h6. testScrubCorruptedCounterRow(org.apache.cassandra.db.ScrubTest):  FAILED
 {noformat}
 Expecting new size of 1, got 2 while replacing 
 [BigTableReader(path='C:\src\refCassandra\build\test\cassandra\data;0\Keyspace1\Counter1-deab62b2d95c11e489c6e117fe147c1d\la-1-big-Data.db')]
  by 
 [BigTableReader(path='C:\src\refCassandra\build\test\cassandra\data;0\Keyspace1\Counter1-deab62b2d95c11e489c6e117fe147c1d\la-1-big-Data.db')]
  in View(pending_count=0, 
 sstables=[BigTableReader(path='C:\src\refCassandra\build\test\cassandra\data;0\Keyspace1\Counter1-deab62b2d95c11e489c6e117fe147c1d\la-3-big-Data.db')],
  compacting=[])
 junit.framework.AssertionFailedError: Expecting new size of 1, got 2 while 
 replacing 
 [BigTableReader(path='C:\src\refCassandra\build\test\cassandra\data;0\Keyspace1\Counter1-deab62b2d95c11e489c6e117fe147c1d\la-1-big-Data.db')]
  by 
 [BigTableReader(path='C:\src\refCassandra\build\test\cassandra\data;0\Keyspace1\Counter1-deab62b2d95c11e489c6e117fe147c1d\la-1-big-Data.db')]
  in View(pending_count=0, 
 sstables=[BigTableReader(path='C:\src\refCassandra\build\test\cassandra\data;0\Keyspace1\Counter1-deab62b2d95c11e489c6e117fe147c1d\la-3-big-Data.db')],
  compacting=[])
at org.apache.cassandra.db.DataTracker$View.replace(DataTracker.java:767)
at org.apache.cassandra.db.DataTracker.replaceReaders(DataTracker.java:408)
at 
 org.apache.cassandra.db.DataTracker.replaceWithNewInstances(DataTracker.java:312)
at 
 org.apache.cassandra.io.sstable.SSTableRewriter.moveStarts(SSTableRewriter.java:341)
at

[1/2] cassandra git commit: Backport CASSANDRA-8243 to 2.0 - Do more aggressive ttl expiration checks to be able to drop more sstables

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 8f2a26c4f - 54ea2f244


Backport CASSANDRA-8243 to 2.0 -  Do more aggressive ttl expiration checks to 
be able to drop more sstables


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13af5978
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13af5978
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13af5978

Branch: refs/heads/cassandra-2.1
Commit: 13af5978225e1ac511fc00a763a8919a2d638dfa
Parents: cefaa4e
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 12:51:44 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 12:58:02 2015 +0200

--
 CHANGES.txt |  2 +
 .../db/compaction/CompactionController.java | 19 ---
 .../cassandra/db/compaction/TTLExpiryTest.java  | 54 ++--
 3 files changed, 62 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3df91ce..22a9515 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.15:
+ * Do more agressive ttl expiration checks to be able to
+   drop more sstables (CASSANDRA-8243)
  * IncomingTcpConnection thread is not named (CASSANDRA-9262)
  * Close incoming connections when MessagingService is stopped (CASSANDRA-9238)
  * Avoid overflow when calculating max sstable size in LCS (CASSANDRA-9235)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index fba659d..7a4b7d9 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -78,12 +78,11 @@ public class CompactionController
  * Finds expired sstables
  *
  * works something like this;
- * 1. find global minTimestamp of overlapping sstables (excluding the 
possibly droppable ones)
- * 2. build a list of candidates to be dropped
- * 3. sort the candidate list, biggest maxTimestamp first in list
- * 4. check if the candidates to be dropped actually can be dropped 
(maxTimestamp  global minTimestamp) and it is included in the compaction
- *- if not droppable, update global minTimestamp and remove from 
candidates
- * 5. return candidates.
+ * 1. find global minTimestamp of overlapping sstables and compacting 
sstables containing any non-expired data
+ * 2. build a list of fully expired candidates
+ * 3. check if the candidates to be dropped actually can be dropped 
(maxTimestamp  global minTimestamp)
+ *- if not droppable, remove from candidates
+ * 4. return candidates.
  *
  * @param cfStore
  * @param compacting we take the drop-candidates from this set, it is 
usually the sstables included in the compaction
@@ -113,10 +112,11 @@ public class CompactionController
 minTimestamp = Math.min(minTimestamp, 
candidate.getMinTimestamp());
 }
 
-// we still need to keep candidates that might shadow something in a
-// non-candidate sstable. And if we remove a sstable from the 
candidates, we
-// must take it's timestamp into account (hence the sorting below).
 Collections.sort(candidates, SSTable.maxTimestampComparator);
+// At this point, minTimestamp denotes the lowest timestamp of any 
relevant
+// SSTable that contains a constructive value. candidates contains all 
the
+// candidates with no constructive values. The ones out of these that 
have
+// (getMaxTimestamp()  minTimestamp) serve no purpose anymore.
 
 IteratorSSTableReader iterator = candidates.iterator();
 while (iterator.hasNext())
@@ -124,7 +124,6 @@ public class CompactionController
 SSTableReader candidate = iterator.next();
 if (candidate.getMaxTimestamp() = minTimestamp)
 {
-minTimestamp = Math.min(candidate.getMinTimestamp(), 
minTimestamp);
 iterator.remove();
 }
 else

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java 
b/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java
index

cassandra git commit: Backport CASSANDRA-8243 to 2.0 - Do more aggressive ttl expiration checks to be able to drop more sstables

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 cefaa4eb7 - 13af59782


Backport CASSANDRA-8243 to 2.0 -  Do more aggressive ttl expiration checks to 
be able to drop more sstables


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13af5978
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13af5978
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13af5978

Branch: refs/heads/cassandra-2.0
Commit: 13af5978225e1ac511fc00a763a8919a2d638dfa
Parents: cefaa4e
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 12:51:44 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 12:58:02 2015 +0200

--
 CHANGES.txt |  2 +
 .../db/compaction/CompactionController.java | 19 ---
 .../cassandra/db/compaction/TTLExpiryTest.java  | 54 ++--
 3 files changed, 62 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3df91ce..22a9515 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.15:
+ * Do more agressive ttl expiration checks to be able to
+   drop more sstables (CASSANDRA-8243)
  * IncomingTcpConnection thread is not named (CASSANDRA-9262)
  * Close incoming connections when MessagingService is stopped (CASSANDRA-9238)
  * Avoid overflow when calculating max sstable size in LCS (CASSANDRA-9235)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index fba659d..7a4b7d9 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -78,12 +78,11 @@ public class CompactionController
  * Finds expired sstables
  *
  * works something like this;
- * 1. find global minTimestamp of overlapping sstables (excluding the 
possibly droppable ones)
- * 2. build a list of candidates to be dropped
- * 3. sort the candidate list, biggest maxTimestamp first in list
- * 4. check if the candidates to be dropped actually can be dropped 
(maxTimestamp  global minTimestamp) and it is included in the compaction
- *- if not droppable, update global minTimestamp and remove from 
candidates
- * 5. return candidates.
+ * 1. find global minTimestamp of overlapping sstables and compacting 
sstables containing any non-expired data
+ * 2. build a list of fully expired candidates
+ * 3. check if the candidates to be dropped actually can be dropped 
(maxTimestamp  global minTimestamp)
+ *- if not droppable, remove from candidates
+ * 4. return candidates.
  *
  * @param cfStore
  * @param compacting we take the drop-candidates from this set, it is 
usually the sstables included in the compaction
@@ -113,10 +112,11 @@ public class CompactionController
 minTimestamp = Math.min(minTimestamp, 
candidate.getMinTimestamp());
 }
 
-// we still need to keep candidates that might shadow something in a
-// non-candidate sstable. And if we remove a sstable from the 
candidates, we
-// must take it's timestamp into account (hence the sorting below).
 Collections.sort(candidates, SSTable.maxTimestampComparator);
+// At this point, minTimestamp denotes the lowest timestamp of any 
relevant
+// SSTable that contains a constructive value. candidates contains all 
the
+// candidates with no constructive values. The ones out of these that 
have
+// (getMaxTimestamp()  minTimestamp) serve no purpose anymore.
 
 IteratorSSTableReader iterator = candidates.iterator();
 while (iterator.hasNext())
@@ -124,7 +124,6 @@ public class CompactionController
 SSTableReader candidate = iterator.next();
 if (candidate.getMaxTimestamp() = minTimestamp)
 {
-minTimestamp = Math.min(candidate.getMinTimestamp(), 
minTimestamp);
 iterator.remove();
 }
 else

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java 
b/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java
index

[1/3] cassandra git commit: Backport CASSANDRA-8243 to 2.0 - Do more aggressive ttl expiration checks to be able to drop more sstables

Repository: cassandra
Updated Branches:
  refs/heads/trunk 01880a470 - f85df3741


Backport CASSANDRA-8243 to 2.0 -  Do more aggressive ttl expiration checks to 
be able to drop more sstables


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13af5978
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13af5978
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13af5978

Branch: refs/heads/trunk
Commit: 13af5978225e1ac511fc00a763a8919a2d638dfa
Parents: cefaa4e
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 12:51:44 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 12:58:02 2015 +0200

--
 CHANGES.txt |  2 +
 .../db/compaction/CompactionController.java | 19 ---
 .../cassandra/db/compaction/TTLExpiryTest.java  | 54 ++--
 3 files changed, 62 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3df91ce..22a9515 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.15:
+ * Do more agressive ttl expiration checks to be able to
+   drop more sstables (CASSANDRA-8243)
  * IncomingTcpConnection thread is not named (CASSANDRA-9262)
  * Close incoming connections when MessagingService is stopped (CASSANDRA-9238)
  * Avoid overflow when calculating max sstable size in LCS (CASSANDRA-9235)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index fba659d..7a4b7d9 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -78,12 +78,11 @@ public class CompactionController
  * Finds expired sstables
  *
  * works something like this;
- * 1. find global minTimestamp of overlapping sstables (excluding the 
possibly droppable ones)
- * 2. build a list of candidates to be dropped
- * 3. sort the candidate list, biggest maxTimestamp first in list
- * 4. check if the candidates to be dropped actually can be dropped 
(maxTimestamp  global minTimestamp) and it is included in the compaction
- *- if not droppable, update global minTimestamp and remove from 
candidates
- * 5. return candidates.
+ * 1. find global minTimestamp of overlapping sstables and compacting 
sstables containing any non-expired data
+ * 2. build a list of fully expired candidates
+ * 3. check if the candidates to be dropped actually can be dropped 
(maxTimestamp  global minTimestamp)
+ *- if not droppable, remove from candidates
+ * 4. return candidates.
  *
  * @param cfStore
  * @param compacting we take the drop-candidates from this set, it is 
usually the sstables included in the compaction
@@ -113,10 +112,11 @@ public class CompactionController
 minTimestamp = Math.min(minTimestamp, 
candidate.getMinTimestamp());
 }
 
-// we still need to keep candidates that might shadow something in a
-// non-candidate sstable. And if we remove a sstable from the 
candidates, we
-// must take it's timestamp into account (hence the sorting below).
 Collections.sort(candidates, SSTable.maxTimestampComparator);
+// At this point, minTimestamp denotes the lowest timestamp of any 
relevant
+// SSTable that contains a constructive value. candidates contains all 
the
+// candidates with no constructive values. The ones out of these that 
have
+// (getMaxTimestamp()  minTimestamp) serve no purpose anymore.
 
 IteratorSSTableReader iterator = candidates.iterator();
 while (iterator.hasNext())
@@ -124,7 +124,6 @@ public class CompactionController
 SSTableReader candidate = iterator.next();
 if (candidate.getMaxTimestamp() = minTimestamp)
 {
-minTimestamp = Math.min(candidate.getMinTimestamp(), 
minTimestamp);
 iterator.remove();
 }
 else

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13af5978/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java 
b/test/unit/org/apache/cassandra/db/compaction/TTLExpiryTest.java
index

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f85df374
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f85df374
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f85df374

Branch: refs/heads/trunk
Commit: f85df37416db31c0b61acbeedfddbdedafc04634
Parents: 01880a4 54ea2f2
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 12:59:48 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 12:59:48 2015 +0200

--

--

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54ea2f24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54ea2f24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54ea2f24

Branch: refs/heads/cassandra-2.1
Commit: 54ea2f244505b7f5b6a623f7e7b8658c712018ac
Parents: 8f2a26c 13af597
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 12:59:41 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 12:59:41 2015 +0200

--

--

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54ea2f24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54ea2f24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54ea2f24

Branch: refs/heads/trunk
Commit: 54ea2f244505b7f5b6a623f7e7b8658c712018ac
Parents: 8f2a26c 13af597
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 30 12:59:41 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Thu Apr 30 12:59:41 2015 +0200

--

--

[jira] [Updated] (CASSANDRA-8243) DTCS can leave time-overlaps, limiting ability to expire entire SSTables

[
https://issues.apache.org/jira/browse/CASSANDRA-8243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Marcus Eriksson updated CASSANDRA-8243:
---
Fix Version/s: 2.0.15

backported to 2.0 in 13af5978225e1ac511fc00a763a8919a2d638dfa - several users
were hit by this

DTCS can leave time-overlaps, limiting ability to expire entire SSTables

Key: CASSANDRA-8243
URL: https://issues.apache.org/jira/browse/CASSANDRA-8243
Project: Cassandra
Issue Type: Bug
Reporter: Björn Hegerfors
Assignee: Björn Hegerfors
Priority: Minor
Labels: compaction, performance
Fix For: 2.1.3, 2.0.15

Attachments: cassandra-trunk-CASSANDRA-8243-aggressiveTTLExpiry.txt,
cassandra-trunk-CASSANDRA-8243-aggressiveTTLExpiry.txt

CASSANDRA-6602 (DTCS) and CASSANDRA-5228 are supposed to be a perfect match
for tables where every value is written with a TTL. DTCS makes sure to keep
old data separate from new data. So shortly after the TTL has passed,
Cassandra should be able to throw away the whole SSTable containing a given
data point.
CASSANDRA-5228 deletes the very oldest SSTables, and only if they don't
overlap (in terms of timestamps) with another SSTable which cannot be deleted.
DTCS however, can't guarantee that SSTables won't overlap (again, in terms of
timestamps). In a test that I ran, every single SSTable overlapped with its
nearest neighbors by a very tiny amount. My reasoning for why this could
happen is that the dumped memtables were already overlapping from the start.
DTCS will never create an overlap where there is none. I surmised that this
happened in my case because I sent parallel writes which must have come out
of order. This was just locally, and out of order writes should be much more
common non-locally.
That means that the SSTable removal optimization may never get a chance to
kick in!
I can see two solutions:
1. Make DTCS split SSTables on time window borders. This will essentially
only be done on a newly dumped memtable once every base_time_seconds.
2. Make TTL SSTable expiry more aggressive. Relax the conditions on which an
SSTable can be dropped completely, of course without affecting any semantics.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9267) Bootstrapping node, nodetool netstats shows all files 100% received immediately


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9267:
---
Fix Version/s: 2.0.x

 Bootstrapping node, nodetool netstats shows all files 100% received 
 immediately
 ---

 Key: CASSANDRA-9267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9267
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Centos
Reporter: Derek Troy-West
Priority: Minor
  Labels: bootstrap, nodetool, streaming
 Fix For: 2.0.x


 Using Cassandra v.2.0.10.
 Following the progress of a new node bootstrapping with 'nodetool netstats' I 
 notice that each file is shown as 100% received immediately.
 i.e.
 {code}
 $ nodetool netstats -h 172.16.11.17
 Mode: JOINING
 Bootstrap 9ec3ab30-eef1-11e4-a7f2-51821d8b858c
 /172.16.11.16
 /172.16.11.12
 Receiving 103 files, 110808118132 bytes total
 /data/cassandra/01/mail/archive/mail-archive-tmp-jb-49-Data.db 
 5388827962/5388827962 bytes(100%) received from /172.16.11.12
 
 /data/cassandra/02/mail/msg_summary_subject_idx/mail-msg_summary_subject_idx-tmp-jb-11-Data.db
  88344414/88344414 bytes(100%) received from /172.16.11.12
 /data/cassandra/02/OpsCenter/pdps/OpsCenter-pdps-tmp-jb-8-Data.db 
 323/323 bytes(100%) received from /172.16.11.12
 /data/cassandra/02/mail/archive/mail-archive-tmp-jb-63-Data.db 
 10470429245/10470429245 bytes(100%) received from /172.16.11.12
 /data/cassandra/03/mail/mail_log/mail-mail_log-tmp-jb-17-Data.db 
 2846783513/2846783513 bytes(100%) received from /172.16.11.12
 {code}
 The node is bootstrapping 200GB in total, and quite quickly/painlessly as far 
 as I can tell, but each file is shown as 100% transferred already.
 Since this is fairly minor I'm happy to hunt it down and patch it myself if 
 desirable. Any pointers where to start appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9267) Bootstrapping node, nodetool netstats shows all files 100% received immediately


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521499#comment-14521499
 ] 

Philip Thompson commented on CASSANDRA-9267:


/cc [~brandon.williams]

 Bootstrapping node, nodetool netstats shows all files 100% received 
 immediately
 ---

 Key: CASSANDRA-9267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9267
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Centos
Reporter: Derek Troy-West
Priority: Minor
  Labels: bootstrap, nodetool, streaming
 Fix For: 2.0.x


 Using Cassandra v.2.0.10.
 Following the progress of a new node bootstrapping with 'nodetool netstats' I 
 notice that each file is shown as 100% received immediately.
 i.e.
 {code}
 $ nodetool netstats -h 172.16.11.17
 Mode: JOINING
 Bootstrap 9ec3ab30-eef1-11e4-a7f2-51821d8b858c
 /172.16.11.16
 /172.16.11.12
 Receiving 103 files, 110808118132 bytes total
 /data/cassandra/01/mail/archive/mail-archive-tmp-jb-49-Data.db 
 5388827962/5388827962 bytes(100%) received from /172.16.11.12
 
 /data/cassandra/02/mail/msg_summary_subject_idx/mail-msg_summary_subject_idx-tmp-jb-11-Data.db
  88344414/88344414 bytes(100%) received from /172.16.11.12
 /data/cassandra/02/OpsCenter/pdps/OpsCenter-pdps-tmp-jb-8-Data.db 
 323/323 bytes(100%) received from /172.16.11.12
 /data/cassandra/02/mail/archive/mail-archive-tmp-jb-63-Data.db 
 10470429245/10470429245 bytes(100%) received from /172.16.11.12
 /data/cassandra/03/mail/mail_log/mail-mail_log-tmp-jb-17-Data.db 
 2846783513/2846783513 bytes(100%) received from /172.16.11.12
 {code}
 The node is bootstrapping 200GB in total, and quite quickly/painlessly as far 
 as I can tell, but each file is shown as 100% transferred already.
 Since this is fairly minor I'm happy to hunt it down and patch it myself if 
 desirable. Any pointers where to start appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9268) forceTerminateAllRepairSessions should notify that the repair session failed instead of finished

2015-04-30 Thread JIRA

Jimmy Mårdell created CASSANDRA-9268:


 Summary: forceTerminateAllRepairSessions should notify that the 
repair session failed instead of finished
 Key: CASSANDRA-9268
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9268
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jimmy Mårdell
Priority: Minor


There is no way for a client to distinguish between a succesful repair and a 
terminated repair (using the JMX call to abort a hanging repair for instance). 
This fools systems that automatically manages repairs.

I think it should be both logged and some notification sent to the client that 
this happened and that the session failed rather than succeeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7032) Improve vnode allocation

2015-04-30 Thread Branimir Lambov (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521484#comment-14521484
]

Branimir Lambov commented on CASSANDRA-7032:

Merged your changes into the
[branch|https://github.com/apache/cassandra/compare/trunk...blambov:7032-vnode-assignment]
and further refactored {{evaluateImprovement}}. It should now be much more
understandable. I have verified that the running time is not significantly
affected and that the results are the same as before (disregarding
insignificant changes caused by floating point rounding).

The {{expandable}} property means that the ownership range of the token can be
expanded. It is not set in the case where currGroup == newGroup, but I swapped
things around in {{findUpdatedReplicationStart}} to make what we do in that
case clearer.

On your other concern, group clumpings quickly create very heavy
underutilization in the individual tokens, which the algorithm will not accept.
A clumping may be a good short-term fix to a heavily overutilized node; I did a
quick test and excluding neighbours of the same group does appear to give
somewhat worse results (e.g. the included long test no longer passes).
Excluding could potentially also cause problems when the number of groups is
close to the replication factor, hence I think we should leave it as it is for
now. It is not hard to change if we want to do so in the future.

Improve vnode allocation

Key: CASSANDRA-7032
URL: https://issues.apache.org/jira/browse/CASSANDRA-7032
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Benedict
Assignee: Branimir Lambov
Labels: performance, vnodes
Fix For: 3.x

Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java,
TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java,
TestVNodeAllocation.java

It's been known for a little while that random vnode allocation causes
hotspots of ownership. It should be possible to improve dramatically on this
with deterministic allocation. I have quickly thrown together a simple greedy
algorithm that allocates vnodes efficiently, and will repair hotspots in a
randomly allocated cluster gradually as more nodes are added, and also
ensures that token ranges are fairly evenly spread between nodes (somewhat
tunably so). The allocation still permits slight discrepancies in ownership,
but it is bound by the inverse of the size of the cluster (as opposed to
random allocation, which strangely gets worse as the cluster size increases).
I'm sure there is a decent dynamic programming solution to this that would be
even better.
If on joining the ring a new node were to CAS a shared table where a
canonical allocation of token ranges lives after running this (or a similar)
algorithm, we could then get guaranteed bounds on the ownership distribution
in a cluster. This will also help for CASSANDRA-6696.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9097) Repeated incremental nodetool repair results in failed repairs due to running anticompaction


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521236#comment-14521236
 ] 

Yuki Morishita commented on CASSANDRA-9097:
---

Patch available here: https://github.com/yukim/cassandra/tree/9097
(This depends on work in CASSANDRA-9151)

In order to fix this in 2.1, I introduced hack that checks {{release_version}} 
of {{system.peers}} table and if it is newer than the version with this fix (I 
make it 2.1.6, but it may need to be adjusted before committing), repair 
waits for response of anti compaction for upto 1 day(this timeout can be 
debatable).

 Repeated incremental nodetool repair results in failed repairs due to running 
 anticompaction
 

 Key: CASSANDRA-9097
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9097
 Project: Cassandra
  Issue Type: Bug
Reporter: Gustav Munkby
Assignee: Yuki Morishita
Priority: Minor
 Fix For: 2.1.x


 I'm trying to synchronize incremental repairs over multiple nodes in a 
 Cassandra cluster, and it does not seem to easily achievable.
 In principle, the process iterates through the nodes of the cluster and 
 performs `nodetool -h $NODE repair --incremental`, but that sometimes fails 
 on subsequent nodes. The reason for failing seems to be that the repair 
 returns as soon as the repair and the _local_ anticompaction has completed, 
 but does not guarantee that remote anticompactions are complete. If I 
 subsequently try to issue another repair command, they fail to start (and 
 terminate with failure after about one minute). It usually isn't a problem, 
 as the local anticompaction typically involves as much (or more) data as the 
 remote ones, but sometimes not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9097) Repeated incremental nodetool repair results in failed repairs due to running anticompaction

2015-04-30 Thread Jonathan Ellis (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-9097:
--
Reviewer: Marcus Eriksson

[~krummas] to review

 Repeated incremental nodetool repair results in failed repairs due to running 
 anticompaction
 

 Key: CASSANDRA-9097
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9097
 Project: Cassandra
  Issue Type: Bug
Reporter: Gustav Munkby
Assignee: Yuki Morishita
Priority: Minor
 Fix For: 2.1.x


 I'm trying to synchronize incremental repairs over multiple nodes in a 
 Cassandra cluster, and it does not seem to easily achievable.
 In principle, the process iterates through the nodes of the cluster and 
 performs `nodetool -h $NODE repair --incremental`, but that sometimes fails 
 on subsequent nodes. The reason for failing seems to be that the repair 
 returns as soon as the repair and the _local_ anticompaction has completed, 
 but does not guarantee that remote anticompactions are complete. If I 
 subsequently try to issue another repair command, they fail to start (and 
 terminate with failure after about one minute). It usually isn't a problem, 
 as the local anticompaction typically involves as much (or more) data as the 
 remote ones, but sometimes not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8897) Remove FileCacheService, instead pooling the buffers

2015-04-30 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521568#comment-14521568
 ] 

Benedict commented on CASSANDRA-8897:
-

[~stefania_alborghetti] I've force pushed a version with a lot of suggestions 
to [the same repository as 
before|https://github.com/belliottsmith/cassandra/tree/8897-suggestions]

There was a reasonably serious bug with concurrent free that I've fixed. We 
should create a stress burn test (see 
[here|https://github.com/belliottsmith/injector/tree/master/src/test/java/bes] 
for some examples, or LongOpOrderTest, or LongSharedExecutorPoolTest) that we 
can leave running for an extended period to be certain of its concurrent 
behaviour.

I also got a bit into the weeds, so I hope you don't feel like I'm treading on 
your toes too much. I started out with the intention of:

* renaming a few things (the terms Chunk were a bit overloaded, for instance, 
and methods like get weren't sufficiently descriptive IMO), 
* trim a few bits of unnecessary boilerplate:
** the Allocator class we only need a single version of, and only one method 
call
** a lot of trace statements for things that should never occur (guess they may 
have been for debugging?) , or error log messages for things that should 
probably be assertions
* simplifying the slot/reclaim logic

Combined that turned out to be a lot, and in the last of these really I went to 
town. I wanted to just make it a little more efficient in the common case, but 
as I thought about it I realised there is a much simpler approach that gives us 
near optimal behaviour in all cases, and is much simpler to reason about, so 
I've half implemented it to confirm it makes sense and is relatively simple and 
clear, and left the mess for you to clean up. There are a lot of potential 
decisions still to be made, though.

The idea is pretty simple: chunk up any given buffer into 64 units, and have a 
single long as a bitmap for which bits are available, and which are not. To 
allocate we now round up to the nearest *unit*, not the nearest power of 2, so 
we can allocate with finer granularity. To find an allocation, we convert it 
into a bitmask that represents the number of units we need, and search in the 
long for the first occurrence of the bitmask that is sufficiently aligned. We 
then clear those bits. On release we simply set the bits again.

The nice thing about this is it has very little code, has guaranteed bounds on 
running time, and no extra memory allocation or compaction of slots. Allocation 
and free are both constant time. We can probably lower the constant factor for 
fragmented allocations if we care to, but I'm not convinced it warrants it.

What I haven't done: 

* currently this means we slice a 64Kb block into 1Kb units. To allocate 
smaller buffers, we can create a microChunk from an allocation within a chunk 
(at the localPool level), from which we can serve smaller requests (which could 
be served in multiples of 16 bytes, so we get finer granularity again). This 
could also help us avoid the problem of wastage if we were to, say, allocate a 
64/32K buffer when we still had 16K spare in the current chunk, since we could 
convert the remainder into a microChunk for serving any small requests.
* We could safely and cheaply assert the buffer has not already been freed
* We could consider making this fully concurrent, dropping the normalFree and 
atomicFree, and just using the bitmap for determining its current status via a 
CAS. I was generally hoping to avoid introducing extra concurrency on the 
critical path, but we could potentially have two paths, one for concurrent and 
one for non-concurrent access, and introduce a flag so that any concurrent free 
on a non-concurrent path would fail. With or without this, though, I like the 
increased simplicity of only relying on the bitmap, since that means only a 
handful of lines of code to understand the memory management
* We could consider making the chunks available for reallocation before they 
are fully free, since there's no different between a partially or fully free 
chunk now for allocation purposes

What I've also done:
* I've conditionally made the attachment a Ref object, so that in unit tests / 
dtests (or other scenarios) we can debug potential leaks of bytebuffers

 Remove FileCacheService, instead pooling the buffers
 

 Key: CASSANDRA-8897
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8897
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Stefania
 Fix For: 3.x


 After CASSANDRA-8893, a RAR will be a very lightweight object and will not 
 need caching, so we can eliminate this cache entirely. Instead we should have 
 a pool of buffers that are

[jira] [Updated] (CASSANDRA-9249) Resetting local schema can cause assertion error


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-9249:
--
Fix Version/s: (was: 2.0.x)
   (was: 2.1.x)
   2.1.6
   2.0.15
   3.0

 Resetting local schema can cause assertion error
 

 Key: CASSANDRA-9249
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9249
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Assignee: T Jake Luciani
Priority: Minor
 Fix For: 3.0, 2.0.15, 2.1.6

 Attachments: 9249fix.txt


 When a user resets their local schema the Schema class purges all CFmetadata 
 and recreates it.  If a compaction is going on this can cause an issue 
 similar to CASSANDRA-8332
 The original intent of the assertion was to ensure if the setLiveMetadata was 
 never called with different metadata instances since the schema is managed as 
 a global reference. However I missed this reset method so it's probably just 
 as well to remove this assert.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8103) Secondary Indices for Static Columns


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8103:
---
 Priority: Blocker  (was: Major)
Fix Version/s: (was: 3.x)
   3.0

I believe this is going to be a blocker for 3.0.  To fully support some 
Thrift-definable schemas in 3.0, we're going to have to support indexes on 
static columns.

 Secondary Indices for Static Columns
 

 Key: CASSANDRA-8103
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8103
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Ron Cohen
Priority: Blocker
 Fix For: 3.0

 Attachments: in_progress.patch


 We should add secondary index support for static columns.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8717) Top-k queries with custom secondary indexes

2015-04-30 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521771#comment-14521771
 ] 

Andrés de la Peña commented on CASSANDRA-8717:
--

I have uploaded a new version of the patch with the changes that need to be 
addressed before including this in 2.1.

As you suggested, I have added a boolean argument named {{trace}} to 
{{SIS#highestSelectivityPredicate}} to indicate whether or not the tracing 
event should be emitted. It is set to true by {{SIS#search}}, and false by 
{{SIS#highestSelectivityIndex}}.

To avoid duplication in {{SecondaryIndexManager}}, now the search method calls 
to {{getHighestSelectivityIndexSearcher}}.

I have modified {{SIS#postReconciliationProcessing}} JavaDoc trying to make it 
clear that it happens on the coordinator node.

I hope you find it OK.

 Top-k queries with custom secondary indexes
 ---

 Key: CASSANDRA-8717
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8717
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Andrés de la Peña
Assignee: Andrés de la Peña
Priority: Minor
  Labels: 2i, secondary_index, sort, sorting, top-k
 Fix For: 3.x

 Attachments: 0001-Add-support-for-top-k-queries-in-2i.patch, 
 0002-Add-support-for-top-k-queries-in-2i.patch, 
 0003-Add-support-for-top-k-queries-in-2i.patch, 
 0004-Add-support-for-top-k-queries-in-2i.patch


 As presented in [Cassandra Summit Europe 
 2014|https://www.youtube.com/watch?v=Hg5s-hXy_-M], secondary indexes can be 
 modified to support general top-k queries with minimum changes in Cassandra 
 codebase. This way, custom 2i implementations could provide relevance search, 
 sorting by columns, etc.
 Top-k queries retrieve the k best results for a certain query. That implies 
 querying the k best rows in each token range and then sort them in order to 
 obtain the k globally best rows. 
 For doing that, we propose two additional methods in class 
 SecondaryIndexSearcher:
 {code:java}
 public boolean requiresFullScan(ListIndexExpression clause)
 {
 return false;
 }
 public ListRow sort(ListIndexExpression clause, ListRow rows)
 {
 return rows;
 }
 {code}
 The first one indicates if a query performed in the index requires querying 
 all the nodes in the ring. It is necessary in top-k queries because we do not 
 know which node are the best results. The second method specifies how to sort 
 all the partial node results according to the query. 
 Then we add two similar methods to the class AbstractRangeCommand:
 {code:java}
 this.searcher = 
 Keyspace.open(keyspace).getColumnFamilyStore(columnFamily).indexManager.searcher(rowFilter);
 public boolean requiresFullScan() {
 return searcher == null ? false : searcher.requiresFullScan(rowFilter);
 }
 public ListRow combine(ListRow rows)
 {
 return searcher == null ? trim(rows) : trim(searcher.sort(rowFilter, 
 rows));
 }
 {code}
 Finnally, we modify StorageProxy#getRangeSlice to use the previous method, as 
 shown in the attached patch.
 We think that the proposed approach provides very useful functionality with 
 minimum impact in current codebase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9269) Huge commitlog not flushed.

2015-04-30 Thread Moloud Shahbazi (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Moloud Shahbazi updated CASSANDRA-9269:
---
Issue Type: Wish  (was: Bug)

 Huge commitlog not flushed.
 ---

 Key: CASSANDRA-9269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9269
 Project: Cassandra
  Issue Type: Wish
 Environment: ubuntu
 cassandra-2.1.3
Reporter: Moloud Shahbazi

 I wrote at once a lot of data in several column families of keyspaceName on a 
 single node cluster. I only read from this keyspace afterward. My issue is 
 that size of commitlog is huge and does not shrink:
 $ du -sh data/*
 7.8Gdata/commitlog
 7.0Gdata/data
 36M data/saved_caches
 When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
 occurs and no change happens in size of commit log.  even when I specify 
 keyspaceName no change happens.
 Also Fot further information, the following is description of my keyspace 
 schema, 
 {code}
 cqlsh describe keyspace keyspaceName;
 CREATE KEYSPACE keyspaceName WITH replication = {'class': 
 'NetworkTopologyStrategy', 'datacenter1': '1'}  AND durable_writes = true;
 CREATE TABLE keyspaceName.CF0 (
 concept_id text,
 chunk_id int,
 json_data text,
 PRIMARY KEY (concept_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF1 (
 item_id text,
 chunk_id int,
 reviews text,
 PRIMARY KEY (item_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF2 (
 review_id text PRIMARY KEY,
 conceptids text,
 score int
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9151) Anti-compaction is blocking ANTI_ENTROPY stage


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522019#comment-14522019
 ] 

Yuki Morishita commented on CASSANDRA-9151:
---

Yes, ANTICOMPATION_REQUEST is for incremental repair in 2.1.
CASSANDRA-8696 can be caused by this, but it's likely caused by timeout waiting 
for snapshot to complete.
Workaround is to temporarily increase rpc_timeout, and we are extending timeout 
in CASSANDRA-9261.

 Anti-compaction is blocking ANTI_ENTROPY stage 
 ---

 Key: CASSANDRA-9151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9151
 Project: Cassandra
  Issue Type: Bug
Reporter: sankalp kohli
Assignee: Yuki Morishita
 Fix For: 3.x, 2.1.x

 Attachments: 
 0001-Do-not-block-stage-when-performing-anticompaction.patch


 In RepairMessageVerbHandler.doVerb case ANTICOMPACTION_REQUEST
 We block on ANTI_ENTROPY stage. This is not good as no other message could be 
 processed till any anticompaction is running. This can cause snapshots for 
 new repairs to fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9269) Huge commitlog not flushed.

2015-04-30 Thread Moloud Shahbazi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521783#comment-14521783
 ] 

Moloud Shahbazi commented on CASSANDRA-9269:


yes but I suppose when memtable is flushed, the write logs in commitlog get 
removed!
Any idea why commitlog stays large?


 Huge commitlog not flushed.
 ---

 Key: CASSANDRA-9269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9269
 Project: Cassandra
  Issue Type: Bug
 Environment: ubuntu
 cassandra-2.1.3
Reporter: Moloud Shahbazi
 Fix For: 2.1.x


 I wrote at once a lot of data in several column families of keyspaceName on a 
 single node cluster. I only read from this keyspace afterward. My issue is 
 that size of commitlog is huge and does not shrink:
 $ du -sh data/*
 7.8Gdata/commitlog
 7.0Gdata/data
 36M data/saved_caches
 When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
 occurs and no change happens in size of commit log.  even when I specify 
 keyspaceName no change happens.
 Also Fot further information, the following is description of my keyspace 
 schema, 
 {code}
 cqlsh describe keyspace keyspaceName;
 CREATE KEYSPACE keyspaceName WITH replication = {'class': 
 'NetworkTopologyStrategy', 'datacenter1': '1'}  AND durable_writes = true;
 CREATE TABLE keyspaceName.CF0 (
 concept_id text,
 chunk_id int,
 json_data text,
 PRIMARY KEY (concept_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF1 (
 item_id text,
 chunk_id int,
 reviews text,
 PRIMARY KEY (item_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF2 (
 review_id text PRIMARY KEY,
 conceptids text,
 score int
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db127a10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db127a10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db127a10

Branch: refs/heads/trunk
Commit: db127a1073cfec426019c06d8f33e6c44292949f
Parents: 80c5191 593a725
Author: Tyler Hobbs tylerlho...@gmail.com
Authored: Thu Apr 30 13:09:11 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Thu Apr 30 13:09:11 2015 -0500

--
 CHANGES.txt |   1 +
 .../cassandra/db/compaction/Scrubber.java   | 127 +++
 .../unit/org/apache/cassandra/SchemaLoader.java |  17 +-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 156 +--
 4 files changed, 256 insertions(+), 45 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/db127a10/CHANGES.txt
--
diff --cc CHANGES.txt
index cb235a4,4e7a5d0..c063368
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,98 -1,11 +1,99 @@@
 -2.0.15:
 +2.1.6
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 +Merged from 2.0:
+  * Allow scrub to handle corrupted compressed chunks (CASSANDRA-9140)
   * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
   * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
 - * Do more agressive ttl expiration checks to be able to
 -   drop more sstables (CASSANDRA-8243)
   * IncomingTcpConnection thread is not named (CASSANDRA-9262)
   * Close incoming connections when MessagingService is stopped 
(CASSANDRA-9238)
 +
 +2.1.5
 + * Re-add deprecated cold_reads_to_omit param for backwards compat 
(CASSANDRA-9203)
 + * Make anticompaction visible in compactionstats (CASSANDRA-9098)
 + * Improve nodetool getendpoints documentation about the partition
 +   key parameter (CASSANDRA-6458)
 + * Don't check other keyspaces for schema changes when an user-defined
 +   type is altered (CASSANDRA-9187)
 + * Allow takeColumnFamilySnapshot to take a list of tables (CASSANDRA-8348)
 + * Limit major sstable operations to their canonical representation 
(CASSANDRA-8669)
 + * cqlsh: Add tests for INSERT and UPDATE tab completion (CASSANDRA-9125)
 + * cqlsh: quote column names when needed in COPY FROM inserts (CASSANDRA-9080)
 + * Add generate-idea-files target to build.xml (CASSANDRA-9123)
 + * Do not load read meter for offline operations (CASSANDRA-9082)
 + * cqlsh: Make CompositeType data readable (CASSANDRA-8919)
 + * cqlsh: Fix display of triggers (CASSANDRA-9081)
 + * Fix NullPointerException when deleting or setting an element by index on
 +   a null list collection (CASSANDRA-9077)
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db127a10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db127a10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db127a10

Branch: refs/heads/cassandra-2.1
Commit: db127a1073cfec426019c06d8f33e6c44292949f
Parents: 80c5191 593a725
Author: Tyler Hobbs tylerlho...@gmail.com
Authored: Thu Apr 30 13:09:11 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Thu Apr 30 13:09:11 2015 -0500

--
 CHANGES.txt |   1 +
 .../cassandra/db/compaction/Scrubber.java   | 127 +++
 .../unit/org/apache/cassandra/SchemaLoader.java |  17 +-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 156 +--
 4 files changed, 256 insertions(+), 45 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/db127a10/CHANGES.txt
--
diff --cc CHANGES.txt
index cb235a4,4e7a5d0..c063368
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,98 -1,11 +1,99 @@@
 -2.0.15:
 +2.1.6
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 +Merged from 2.0:
+  * Allow scrub to handle corrupted compressed chunks (CASSANDRA-9140)
   * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
   * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
 - * Do more agressive ttl expiration checks to be able to
 -   drop more sstables (CASSANDRA-8243)
   * IncomingTcpConnection thread is not named (CASSANDRA-9262)
   * Close incoming connections when MessagingService is stopped 
(CASSANDRA-9238)
 +
 +2.1.5
 + * Re-add deprecated cold_reads_to_omit param for backwards compat 
(CASSANDRA-9203)
 + * Make anticompaction visible in compactionstats (CASSANDRA-9098)
 + * Improve nodetool getendpoints documentation about the partition
 +   key parameter (CASSANDRA-6458)
 + * Don't check other keyspaces for schema changes when an user-defined
 +   type is altered (CASSANDRA-9187)
 + * Allow takeColumnFamilySnapshot to take a list of tables (CASSANDRA-8348)
 + * Limit major sstable operations to their canonical representation 
(CASSANDRA-8669)
 + * cqlsh: Add tests for INSERT and UPDATE tab completion (CASSANDRA-9125)
 + * cqlsh: quote column names when needed in COPY FROM inserts (CASSANDRA-9080)
 + * Add generate-idea-files target to build.xml (CASSANDRA-9123)
 + * Do not load read meter for offline operations (CASSANDRA-9082)
 + * cqlsh: Make CompositeType data readable (CASSANDRA-8919)
 + * cqlsh: Fix display of triggers (CASSANDRA-9081)
 + * Fix NullPointerException when deleting or setting an element by index on
 +   a null list collection (CASSANDRA-9077)
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the

[1/2] cassandra git commit: Handle corrupted compression chunks in scrub

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 80c51919b - db127a107


Handle corrupted compression chunks in scrub

Patch by Stefania Alborghetti; reviewed by Tyler Hobbs for
CASSANDRA-9140


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/593a7257
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/593a7257
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/593a7257

Branch: refs/heads/cassandra-2.1
Commit: 593a7257b5d5243c4ceb75b5417fb05e8a98cc90
Parents: a85e731
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Thu Apr 30 13:02:23 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Thu Apr 30 13:02:23 2015 -0500

--
 CHANGES.txt |   1 +
 .../cassandra/db/compaction/Scrubber.java   | 131 
 .../unit/org/apache/cassandra/SchemaLoader.java |  17 ++-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 153 +--
 4 files changed, 256 insertions(+), 46 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/593a7257/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9f90cc2..4e7a5d0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Allow scrub to handle corrupted compressed chunks (CASSANDRA-9140)
  * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to

http://git-wip-us.apache.org/repos/asf/cassandra/blob/593a7257/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java 
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index 6a61e56..1752b21 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -21,6 +21,7 @@ import java.nio.ByteBuffer;
 import java.io.*;
 import java.util.*;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Throwables;
 
 import org.apache.cassandra.db.*;
@@ -53,6 +54,11 @@ public class Scrubber implements Closeable
 private int badRows;
 private int emptyRows;
 
+private ByteBuffer currentIndexKey;
+private ByteBuffer nextIndexKey;
+long currentRowPositionFromIndex;
+long nextRowPositionFromIndex;
+
 private final OutputHandler outputHandler;
 
 private static final ComparatorRow rowComparator = new ComparatorRow()
@@ -99,6 +105,9 @@ public class Scrubber implements Closeable
 : 
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
 this.indexFile = RandomAccessReader.open(new 
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
 this.scrubInfo = new ScrubInfo(dataFile, sstable);
+
+this.currentRowPositionFromIndex = 0;
+this.nextRowPositionFromIndex = 0;
 }
 
 public void scrub()
@@ -106,7 +115,7 @@ public class Scrubber implements Closeable
 outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable, 
dataFile.length()));
 try
 {
-ByteBuffer nextIndexKey = 
ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
 {
 // throw away variable so we don't have a side effect in the 
assert
 long firstRowPositionFromIndex = 
RowIndexEntry.serializer.deserialize(indexFile, 
sstable.descriptor.version).position;
@@ -122,6 +131,7 @@ public class Scrubber implements Closeable
 {
 if (scrubInfo.isStopRequested())
 throw new 
CompactionInterruptedException(scrubInfo.getCompactionInfo());
+
 long rowStart = dataFile.getFilePointer();
 outputHandler.debug(Reading row at  + rowStart);
 
@@ -142,29 +152,19 @@ public class Scrubber implements Closeable
 // check for null key below
 }
 
-ByteBuffer currentIndexKey = nextIndexKey;
-long nextRowPositionFromIndex;
-try
-{
-nextIndexKey = indexFile.isEOF() ? null : 
ByteBufferUtil.readWithShortLength(indexFile);
-nextRowPositionFromIndex = indexFile.isEOF()
- ? dataFile.length()
- : 
RowIndexEntry.serializer.deserialize(indexFile, 
sstable.descriptor.version).position;
-

cassandra git commit: Handle corrupted compression chunks in scrub

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 a85e731f2 - 593a7257b


Handle corrupted compression chunks in scrub

Patch by Stefania Alborghetti; reviewed by Tyler Hobbs for
CASSANDRA-9140


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/593a7257
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/593a7257
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/593a7257

Branch: refs/heads/cassandra-2.0
Commit: 593a7257b5d5243c4ceb75b5417fb05e8a98cc90
Parents: a85e731
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Thu Apr 30 13:02:23 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Thu Apr 30 13:02:23 2015 -0500

--
 CHANGES.txt |   1 +
 .../cassandra/db/compaction/Scrubber.java   | 131 
 .../unit/org/apache/cassandra/SchemaLoader.java |  17 ++-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 153 +--
 4 files changed, 256 insertions(+), 46 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/593a7257/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9f90cc2..4e7a5d0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Allow scrub to handle corrupted compressed chunks (CASSANDRA-9140)
  * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to

http://git-wip-us.apache.org/repos/asf/cassandra/blob/593a7257/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java 
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index 6a61e56..1752b21 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -21,6 +21,7 @@ import java.nio.ByteBuffer;
 import java.io.*;
 import java.util.*;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Throwables;
 
 import org.apache.cassandra.db.*;
@@ -53,6 +54,11 @@ public class Scrubber implements Closeable
 private int badRows;
 private int emptyRows;
 
+private ByteBuffer currentIndexKey;
+private ByteBuffer nextIndexKey;
+long currentRowPositionFromIndex;
+long nextRowPositionFromIndex;
+
 private final OutputHandler outputHandler;
 
 private static final ComparatorRow rowComparator = new ComparatorRow()
@@ -99,6 +105,9 @@ public class Scrubber implements Closeable
 : 
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
 this.indexFile = RandomAccessReader.open(new 
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
 this.scrubInfo = new ScrubInfo(dataFile, sstable);
+
+this.currentRowPositionFromIndex = 0;
+this.nextRowPositionFromIndex = 0;
 }
 
 public void scrub()
@@ -106,7 +115,7 @@ public class Scrubber implements Closeable
 outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable, 
dataFile.length()));
 try
 {
-ByteBuffer nextIndexKey = 
ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
 {
 // throw away variable so we don't have a side effect in the 
assert
 long firstRowPositionFromIndex = 
RowIndexEntry.serializer.deserialize(indexFile, 
sstable.descriptor.version).position;
@@ -122,6 +131,7 @@ public class Scrubber implements Closeable
 {
 if (scrubInfo.isStopRequested())
 throw new 
CompactionInterruptedException(scrubInfo.getCompactionInfo());
+
 long rowStart = dataFile.getFilePointer();
 outputHandler.debug(Reading row at  + rowStart);
 
@@ -142,29 +152,19 @@ public class Scrubber implements Closeable
 // check for null key below
 }
 
-ByteBuffer currentIndexKey = nextIndexKey;
-long nextRowPositionFromIndex;
-try
-{
-nextIndexKey = indexFile.isEOF() ? null : 
ByteBufferUtil.readWithShortLength(indexFile);
-nextRowPositionFromIndex = indexFile.isEOF()
- ? dataFile.length()
- : 
RowIndexEntry.serializer.deserialize(indexFile, 
sstable.descriptor.version).position;
-

[1/2] cassandra git commit: Fix AssertionError when resetlocalschema is run during repair

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 1f6bf3651 - 80c51919b


Fix AssertionError when resetlocalschema is run during repair

Patch by tjake; reviewed by carlyeks for CASSANDRA-9249


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a85e731f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a85e731f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a85e731f

Branch: refs/heads/cassandra-2.1
Commit: a85e731f297236f6f43ea95c3549f2d1424926fa
Parents: 75e858a
Author: T Jake Luciani j...@apache.org
Authored: Thu Apr 30 12:36:32 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Apr 30 12:39:33 2015 -0400

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/io/compress/CompressionParameters.java | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a85e731f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8a3e5a9..9f90cc2 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to
drop more sstables (CASSANDRA-8243)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a85e731f/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java 
b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
index 98f01d4..cf16711 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
@@ -108,7 +108,6 @@ public class CompressionParameters
 if (liveMetadata == null)
 return;
 
-assert this.liveMetadata == null || this.liveMetadata == liveMetadata;
 this.liveMetadata = liveMetadata;
 }

[jira] [Updated] (CASSANDRA-9269) Huge commitlog not flushed.

2015-04-30 Thread mullan (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mullan updated CASSANDRA-9269:
--
Description: 
I wrote at once a lot of data in several column families of keyspaceName on a 
single node cluster. I only read from this keyspace afterward. My issue is that 
size of commitlog is huge and does not shrink:
$ du -sh data/*
7.8Gdata/commitlog
7.0Gdata/data
36M data/saved_caches

When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
occurs and no change happens in size of commit log.  even when I specify 
keyspaceName no change happens.

  was:
I wrote at once a lot of data in several column families of keyspaceName on a 
single node cluster. I only read from this keyspace afterward. My issue is that 
size of commitlog is huge and does not shrink:
$ du -sh data/*
7.8Gdata/commitlog
7.0Gdata/data
36M data/saved_caches

When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
occurs and no change happens in size of commit log.  even when I specify 
keyspaceName no change happens.

Also Fot further information, the following is description of my keyspace 
schema, 
{code}
cqlsh describe keyspace keyspaceName;

CREATE KEYSPACE keyspaceName WITH replication = {'class': 
'NetworkTopologyStrategy', 'datacenter1': '1'}  AND durable_writes = true;

CREATE TABLE keyspaceName.CF0 (
concept_id text,
chunk_id int,
json_data text,
PRIMARY KEY (concept_id, chunk_id)
) WITH CLUSTERING ORDER BY (chunk_id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

CREATE TABLE keyspaceName.CF1 (
item_id text,
chunk_id int,
reviews text,
PRIMARY KEY (item_id, chunk_id)
) WITH CLUSTERING ORDER BY (chunk_id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

CREATE TABLE keyspaceName.CF2 (
review_id text PRIMARY KEY,
conceptids text,
score int
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';{code}


 Huge commitlog not flushed.
 ---

 Key: CASSANDRA-9269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9269
 Project: Cassandra
  Issue Type: Wish
 Environment: ubuntu
 cassandra-2.1.3
Reporter: mullan
Priority: Trivial

 I wrote at once a lot of data in several column families of keyspaceName on a 
 single node cluster. I only read from this keyspace afterward. My issue is 
 that size of commitlog is huge and does not shrink:
 $ du -sh data/*
 7.8Gdata/commitlog
 7.0Gdata/data
 36M data/saved_caches
 When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
 occurs and no change happens in size of commit log.  even when I specify 
 keyspaceName no change happens.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8099) Refactor and modernize the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

T Jake Luciani updated CASSANDRA-8099:
--
Fix Version/s: (was: 3.x)
3.0

Refactor and modernize the storage engine
-

Key: CASSANDRA-8099
URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 3.0

Attachments: 8099-nit

The current storage engine (which for this ticket I'll loosely define as the
code implementing the read/write path) is suffering from old age. One of the
main problem is that the only structure it deals with is the cell, which
completely ignores the more high level CQL structure that groups cell into
(CQL) rows.
This leads to many inefficiencies, like the fact that during a reads we have
to group cells multiple times (to count on replica, then to count on the
coordinator, then to produce the CQL resultset) because we forget about the
grouping right away each time (so lots of useless cell names comparisons in
particular). But outside inefficiencies, having to manually recreate the CQL
structure every time we need it for something is hindering new features and
makes the code more complex that it should be.
Said storage engine also has tons of technical debt. To pick an example, the
fact that during range queries we update {{SliceQueryFilter.count}} is pretty
hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has
to go into to simply remove the last query result.
So I want to bite the bullet and modernize this storage engine. I propose to
do 2 main things:
# Make the storage engine more aware of the CQL structure. In practice,
instead of having partitions be a simple iterable map of cells, it should be
an iterable list of row (each being itself composed of per-column cells,
though obviously not exactly the same kind of cell we have today).
# Make the engine more iterative. What I mean here is that in the read path,
we end up reading all cells in memory (we put them in a ColumnFamily object),
but there is really no reason to. If instead we were working with iterators
all the way through, we could get to a point where we're basically
transferring data from disk to the network, and we should be able to reduce
GC substantially.
Please note that such refactor should provide some performance improvements
right off the bat but it's not it's primary goal either. It's primary goal is
to simplify the storage engine and adds abstraction that are better suited to
further optimizations.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9271) IndexSummaryManagerTest.testCompactionRace times out periodically

Ariel Weisberg created CASSANDRA-9271:
-

 Summary: IndexSummaryManagerTest.testCompactionRace times out 
periodically
 Key: CASSANDRA-9271
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9271
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg


The issue is that the amount of time the test takes is highly variable to it 
being biased towards creating a condition where the test has to retry the 
compaction it is attempting.

Solution is to decrease the bias by having 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L2522
 check every millisecond instead of every 100 milliseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/80c51919
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/80c51919
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/80c51919

Branch: refs/heads/cassandra-2.1
Commit: 80c51919b9bc5d20467c7c0bace982e16c30daac
Parents: 1f6bf36 a85e731
Author: T Jake Luciani j...@apache.org
Authored: Thu Apr 30 12:41:28 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Apr 30 12:41:28 2015 -0400

--
 CHANGES.txt| 2 +-
 .../org/apache/cassandra/io/compress/CompressionParameters.java| 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/80c51919/CHANGES.txt
--
diff --cc CHANGES.txt
index a01e8ed,9f90cc2..cb235a4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,98 -1,10 +1,98 @@@
 -2.0.15:
 +2.1.6
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 +Merged from 2.0:
+  * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
   * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
 - * Do more agressive ttl expiration checks to be able to
 -   drop more sstables (CASSANDRA-8243)
   * IncomingTcpConnection thread is not named (CASSANDRA-9262)
   * Close incoming connections when MessagingService is stopped 
(CASSANDRA-9238)
 +
- 
 +2.1.5
 + * Re-add deprecated cold_reads_to_omit param for backwards compat 
(CASSANDRA-9203)
 + * Make anticompaction visible in compactionstats (CASSANDRA-9098)
 + * Improve nodetool getendpoints documentation about the partition
 +   key parameter (CASSANDRA-6458)
 + * Don't check other keyspaces for schema changes when an user-defined
 +   type is altered (CASSANDRA-9187)
 + * Allow takeColumnFamilySnapshot to take a list of tables (CASSANDRA-8348)
 + * Limit major sstable operations to their canonical representation 
(CASSANDRA-8669)
 + * cqlsh: Add tests for INSERT and UPDATE tab completion (CASSANDRA-9125)
 + * cqlsh: quote column names when needed in COPY FROM inserts (CASSANDRA-9080)
 + * Add generate-idea-files target to build.xml (CASSANDRA-9123)
 + * Do not load read meter for offline operations (CASSANDRA-9082)
 + * cqlsh: Make CompositeType data readable (CASSANDRA-8919)
 + * cqlsh: Fix display of triggers (CASSANDRA-9081)
 + * Fix NullPointerException when deleting or setting an element by index on
 +   a null list collection (CASSANDRA-9077)
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation

[jira] [Commented] (CASSANDRA-9151) Anti-compaction is blocking ANTI_ENTROPY stage


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521901#comment-14521901
 ] 

Aleksey Yeschenko commented on CASSANDRA-9151:
--

[~kohlisankalp] can you review soonish?

 Anti-compaction is blocking ANTI_ENTROPY stage 
 ---

 Key: CASSANDRA-9151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9151
 Project: Cassandra
  Issue Type: Bug
Reporter: sankalp kohli
Assignee: Yuki Morishita
 Fix For: 3.x, 2.1.x

 Attachments: 
 0001-Do-not-block-stage-when-performing-anticompaction.patch


 In RepairMessageVerbHandler.doVerb case ANTICOMPACTION_REQUEST
 We block on ANTI_ENTROPY stage. This is not good as no other message could be 
 processed till any anticompaction is running. This can cause snapshots for 
 new repairs to fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9151) Anti-compaction is blocking ANTI_ENTROPY stage

2015-04-30 Thread Randy Fradin (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521917#comment-14521917
 ] 

Randy Fradin commented on CASSANDRA-9151:
-

Are ANTICOMPACTION_REQUEST messages only sent when doing incremental repair? 
I've seen Could not create snapshot in the logs like other users in 
CASSANDRA-8696, but we've only been using full repair. Trying to understand if 
this bug could still be the cause of this issue.

 Anti-compaction is blocking ANTI_ENTROPY stage 
 ---

 Key: CASSANDRA-9151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9151
 Project: Cassandra
  Issue Type: Bug
Reporter: sankalp kohli
Assignee: Yuki Morishita
 Fix For: 3.x, 2.1.x

 Attachments: 
 0001-Do-not-block-stage-when-performing-anticompaction.patch


 In RepairMessageVerbHandler.doVerb case ANTICOMPACTION_REQUEST
 We block on ANTI_ENTROPY stage. This is not good as no other message could be 
 processed till any anticompaction is running. This can cause snapshots for 
 new repairs to fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9266) Repair failed with error Already repairing SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'), can not continue.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521777#comment-14521777
 ] 

Marcus Eriksson commented on CASSANDRA-9266:


I would need the server logs, from when you first ran repair preferably. You 
can grep for exceptions and post the bits around them if you like

 Repair failed with error Already repairing 
 SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'),
  can not continue.
 ---

 Key: CASSANDRA-9266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9266
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.3
Reporter: Razi Khaja
 Fix For: 2.1.x


 I am running incremental parallel repair using the following command:
 {code}
 nodetool --host my_hostname repair --incremental --in-local-dc --parallel
 {code}
 and get the following error:
 {code}
 Repair failed with error Already repairing 
 SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'),
  can not continue.
 {code}
 I have 3 data centers, each with 4 nodes. Neither incremental or full repair 
 is running on any of my other nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Fix AssertionError when resetlocalschema is run during repair

Repository: cassandra
Updated Branches:
  refs/heads/trunk 774bd0bce - 3bfe4b6ae


Fix AssertionError when resetlocalschema is run during repair

Patch by tjake; reviewed by carlyeks for CASSANDRA-9249


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a85e731f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a85e731f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a85e731f

Branch: refs/heads/trunk
Commit: a85e731f297236f6f43ea95c3549f2d1424926fa
Parents: 75e858a
Author: T Jake Luciani j...@apache.org
Authored: Thu Apr 30 12:36:32 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Apr 30 12:39:33 2015 -0400

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/io/compress/CompressionParameters.java | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a85e731f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8a3e5a9..9f90cc2 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to
drop more sstables (CASSANDRA-8243)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a85e731f/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java 
b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
index 98f01d4..cf16711 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
@@ -108,7 +108,6 @@ public class CompressionParameters
 if (liveMetadata == null)
 return;
 
-assert this.liveMetadata == null || this.liveMetadata == liveMetadata;
 this.liveMetadata = liveMetadata;
 }

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3bfe4b6a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3bfe4b6a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3bfe4b6a

Branch: refs/heads/trunk
Commit: 3bfe4b6ae849add1c3388db3874bbdfd27443822
Parents: 774bd0b 80c5191
Author: T Jake Luciani j...@apache.org
Authored: Thu Apr 30 12:42:10 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Apr 30 12:42:10 2015 -0400

--
 CHANGES.txt| 2 +-
 .../org/apache/cassandra/io/compress/CompressionParameters.java| 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3bfe4b6a/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3bfe4b6a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--

[jira] [Commented] (CASSANDRA-7486) Compare CMS and G1 pause times


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521805#comment-14521805
 ] 

Ariel Weisberg commented on CASSANDRA-7486:
---

I would just like to see the data visualized. If it's not better in every 
dimension then in what dimensions is better/worse in across all the data that 
Al collected.

 Compare CMS and G1 pause times
 --

 Key: CASSANDRA-7486
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
 Project: Cassandra
  Issue Type: Test
  Components: Config
Reporter: Jonathan Ellis
Assignee: Shawn Kumar
 Fix For: 2.1.x


 See 
 http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
  and https://twitter.com/rbranson/status/482113561431265281
 May want to default 2.1 to G1.
 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
 Suspect this will help G1 even more than CMS.  (NB this is off by default but 
 needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Handle corrupted compression chunks in scrub

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3bfe4b6ae - 49b089893


Handle corrupted compression chunks in scrub

Patch by Stefania Alborghetti; reviewed by Tyler Hobbs for
CASSANDRA-9140


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/593a7257
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/593a7257
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/593a7257

Branch: refs/heads/trunk
Commit: 593a7257b5d5243c4ceb75b5417fb05e8a98cc90
Parents: a85e731
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Thu Apr 30 13:02:23 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Thu Apr 30 13:02:23 2015 -0500

--
 CHANGES.txt |   1 +
 .../cassandra/db/compaction/Scrubber.java   | 131 
 .../unit/org/apache/cassandra/SchemaLoader.java |  17 ++-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 153 +--
 4 files changed, 256 insertions(+), 46 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/593a7257/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9f90cc2..4e7a5d0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Allow scrub to handle corrupted compressed chunks (CASSANDRA-9140)
  * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to

http://git-wip-us.apache.org/repos/asf/cassandra/blob/593a7257/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java 
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index 6a61e56..1752b21 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -21,6 +21,7 @@ import java.nio.ByteBuffer;
 import java.io.*;
 import java.util.*;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Throwables;
 
 import org.apache.cassandra.db.*;
@@ -53,6 +54,11 @@ public class Scrubber implements Closeable
 private int badRows;
 private int emptyRows;
 
+private ByteBuffer currentIndexKey;
+private ByteBuffer nextIndexKey;
+long currentRowPositionFromIndex;
+long nextRowPositionFromIndex;
+
 private final OutputHandler outputHandler;
 
 private static final ComparatorRow rowComparator = new ComparatorRow()
@@ -99,6 +105,9 @@ public class Scrubber implements Closeable
 : 
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
 this.indexFile = RandomAccessReader.open(new 
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
 this.scrubInfo = new ScrubInfo(dataFile, sstable);
+
+this.currentRowPositionFromIndex = 0;
+this.nextRowPositionFromIndex = 0;
 }
 
 public void scrub()
@@ -106,7 +115,7 @@ public class Scrubber implements Closeable
 outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable, 
dataFile.length()));
 try
 {
-ByteBuffer nextIndexKey = 
ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
 {
 // throw away variable so we don't have a side effect in the 
assert
 long firstRowPositionFromIndex = 
RowIndexEntry.serializer.deserialize(indexFile, 
sstable.descriptor.version).position;
@@ -122,6 +131,7 @@ public class Scrubber implements Closeable
 {
 if (scrubInfo.isStopRequested())
 throw new 
CompactionInterruptedException(scrubInfo.getCompactionInfo());
+
 long rowStart = dataFile.getFilePointer();
 outputHandler.debug(Reading row at  + rowStart);
 
@@ -142,29 +152,19 @@ public class Scrubber implements Closeable
 // check for null key below
 }
 
-ByteBuffer currentIndexKey = nextIndexKey;
-long nextRowPositionFromIndex;
-try
-{
-nextIndexKey = indexFile.isEOF() ? null : 
ByteBufferUtil.readWithShortLength(indexFile);
-nextRowPositionFromIndex = indexFile.isEOF()
- ? dataFile.length()
- : 
RowIndexEntry.serializer.deserialize(indexFile, 
sstable.descriptor.version).position;
-}
-

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49b08989
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49b08989
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49b08989

Branch: refs/heads/trunk
Commit: 49b089893b5407ba42dad389804ff21d535a8537
Parents: 3bfe4b6 db127a1
Author: Tyler Hobbs tylerlho...@gmail.com
Authored: Thu Apr 30 13:15:57 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Thu Apr 30 13:15:57 2015 -0500

--
 CHANGES.txt |   1 +
 .../cassandra/db/compaction/Scrubber.java   | 128 ---
 .../compress/CompressedRandomAccessReader.java  |   2 +
 .../unit/org/apache/cassandra/SchemaLoader.java |   7 +-
 .../unit/org/apache/cassandra/db/ScrubTest.java | 157 ---
 5 files changed, 242 insertions(+), 53 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49b08989/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/49b08989/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/Scrubber.java
index 675b604,1f5c7de..1e50da6
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@@ -121,10 -122,10 +130,10 @@@ public class Scrubber implements Closea
  SSTableRewriter writer = new SSTableRewriter(cfs, oldSSTable, 
sstable.maxDataAge, isOffline);
  try
  {
- ByteBuffer nextIndexKey = 
ByteBufferUtil.readWithShortLength(indexFile);
+ nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
  {
  // throw away variable so we don't have a side effect in the 
assert
 -long firstRowPositionFromIndex = 
sstable.metadata.comparator.rowIndexEntrySerializer().deserialize(indexFile, 
sstable.descriptor.version).position;
 +long firstRowPositionFromIndex = 
rowIndexEntrySerializer.deserialize(indexFile, 
sstable.descriptor.version).position;
  assert firstRowPositionFromIndex == 0 : 
firstRowPositionFromIndex;
  }
  
@@@ -150,30 -153,19 +160,19 @@@
  // check for null key below
  }
  
- ByteBuffer currentIndexKey = nextIndexKey;
- long nextRowPositionFromIndex;
- try
- {
- nextIndexKey = indexFile.isEOF() ? null : 
ByteBufferUtil.readWithShortLength(indexFile);
- nextRowPositionFromIndex = indexFile.isEOF()
-  ? dataFile.length()
-  : 
rowIndexEntrySerializer.deserialize(indexFile, 
sstable.descriptor.version).position;
- }
- catch (Throwable th)
- {
- JVMStabilityInspector.inspectThrowable(th);
- outputHandler.warn(Error reading index file, th);
- nextIndexKey = null;
- nextRowPositionFromIndex = dataFile.length();
- }
+ updateIndexKey();
  
  long dataStart = dataFile.getFilePointer();
- long dataStartFromIndex = currentIndexKey == null
- ? -1
- : rowStart + 2 + 
currentIndexKey.remaining();
- long dataSizeFromIndex = nextRowPositionFromIndex - 
dataStartFromIndex;
+ 
+ long dataStartFromIndex = -1;
+ long dataSizeFromIndex = -1;
+ if (currentIndexKey != null)
+ {
+ dataStartFromIndex = currentRowPositionFromIndex + 2 + 
currentIndexKey.remaining();
+ dataSizeFromIndex = nextRowPositionFromIndex - 
dataStartFromIndex;
+ }
  
 -dataSize = dataSizeFromIndex;
 +long dataSize = dataSizeFromIndex;
  // avoid an NPE if key is null
  String keyName = key == null ? (unreadable key) : 
ByteBufferUtil.bytesToHex(key.getKey());
  outputHandler.debug(String.format(row %s is %s bytes, 
keyName, dataSize));
@@@ -187,7 -186,13 +193,14 @@@
  if (dataSize  dataFile.length())
  throw new IOError(new IOException(Impossible row 
size  + dataSize));
  
+ if (dataStart != dataStartFromIndex)
+ outputHandler.warn(String.format(Data file row 
position

cassandra git commit: Fix AssertionError when resetlocalschema is run during repair

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 75e858a66 - a85e731f2


Fix AssertionError when resetlocalschema is run during repair

Patch by tjake; reviewed by carlyeks for CASSANDRA-9249


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a85e731f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a85e731f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a85e731f

Branch: refs/heads/cassandra-2.0
Commit: a85e731f297236f6f43ea95c3549f2d1424926fa
Parents: 75e858a
Author: T Jake Luciani j...@apache.org
Authored: Thu Apr 30 12:36:32 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Apr 30 12:39:33 2015 -0400

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/io/compress/CompressionParameters.java | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a85e731f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8a3e5a9..9f90cc2 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.15:
+ * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
  * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
  * Do more agressive ttl expiration checks to be able to
drop more sstables (CASSANDRA-8243)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a85e731f/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git 
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java 
b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
index 98f01d4..cf16711 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
@@ -108,7 +108,6 @@ public class CompressionParameters
 if (liveMetadata == null)
 return;
 
-assert this.liveMetadata == null || this.liveMetadata == liveMetadata;
 this.liveMetadata = liveMetadata;
 }

[jira] [Commented] (CASSANDRA-9269) Huge commitlog not flushed.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521790#comment-14521790
 ] 

Philip Thompson commented on CASSANDRA-9269:


The default commitlog size is 8GB, so this is all working as intended. As new 
writes come in, segments of the commitlog that contain flushed writes will be 
recycled and reused to log new data. If you have questions about how to alter 
the default behavior, email the Cassandra user mailing list, or ask on freenode 
in the #cassandra channel.

Thank you for the thorough bug report.

 Huge commitlog not flushed.
 ---

 Key: CASSANDRA-9269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9269
 Project: Cassandra
  Issue Type: Bug
 Environment: ubuntu
 cassandra-2.1.3
Reporter: Moloud Shahbazi

 I wrote at once a lot of data in several column families of keyspaceName on a 
 single node cluster. I only read from this keyspace afterward. My issue is 
 that size of commitlog is huge and does not shrink:
 $ du -sh data/*
 7.8Gdata/commitlog
 7.0Gdata/data
 36M data/saved_caches
 When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
 occurs and no change happens in size of commit log.  even when I specify 
 keyspaceName no change happens.
 Also Fot further information, the following is description of my keyspace 
 schema, 
 {code}
 cqlsh describe keyspace keyspaceName;
 CREATE KEYSPACE keyspaceName WITH replication = {'class': 
 'NetworkTopologyStrategy', 'datacenter1': '1'}  AND durable_writes = true;
 CREATE TABLE keyspaceName.CF0 (
 concept_id text,
 chunk_id int,
 json_data text,
 PRIMARY KEY (concept_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF1 (
 item_id text,
 chunk_id int,
 reviews text,
 PRIMARY KEY (item_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF2 (
 review_id text PRIMARY KEY,
 conceptids text,
 score int
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9269) Huge commitlog not flushed.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson resolved CASSANDRA-9269.

   Resolution: Not A Problem
Fix Version/s: (was: 2.1.x)

 Huge commitlog not flushed.
 ---

 Key: CASSANDRA-9269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9269
 Project: Cassandra
  Issue Type: Bug
 Environment: ubuntu
 cassandra-2.1.3
Reporter: Moloud Shahbazi

 I wrote at once a lot of data in several column families of keyspaceName on a 
 single node cluster. I only read from this keyspace afterward. My issue is 
 that size of commitlog is huge and does not shrink:
 $ du -sh data/*
 7.8Gdata/commitlog
 7.0Gdata/data
 36M data/saved_caches
 When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
 occurs and no change happens in size of commit log.  even when I specify 
 keyspaceName no change happens.
 Also Fot further information, the following is description of my keyspace 
 schema, 
 {code}
 cqlsh describe keyspace keyspaceName;
 CREATE KEYSPACE keyspaceName WITH replication = {'class': 
 'NetworkTopologyStrategy', 'datacenter1': '1'}  AND durable_writes = true;
 CREATE TABLE keyspaceName.CF0 (
 concept_id text,
 chunk_id int,
 json_data text,
 PRIMARY KEY (concept_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF1 (
 item_id text,
 chunk_id int,
 reviews text,
 PRIMARY KEY (item_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF2 (
 review_id text PRIMARY KEY,
 conceptids text,
 score int
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/80c51919
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/80c51919
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/80c51919

Branch: refs/heads/trunk
Commit: 80c51919b9bc5d20467c7c0bace982e16c30daac
Parents: 1f6bf36 a85e731
Author: T Jake Luciani j...@apache.org
Authored: Thu Apr 30 12:41:28 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Apr 30 12:41:28 2015 -0400

--
 CHANGES.txt| 2 +-
 .../org/apache/cassandra/io/compress/CompressionParameters.java| 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/80c51919/CHANGES.txt
--
diff --cc CHANGES.txt
index a01e8ed,9f90cc2..cb235a4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,98 -1,10 +1,98 @@@
 -2.0.15:
 +2.1.6
 + * Fix PITR commitlog replay (CASSANDRA-9195)
 + * GCInspector logs very different times (CASSANDRA-9124)
 + * Fix deleting from an empty list (CASSANDRA-9198)
 + * Update tuple and collection types that use a user-defined type when that 
UDT
 +   is modified (CASSANDRA-9148, CASSANDRA-9192)
 +Merged from 2.0:
+  * Fix assertion error when resetlocalschema is run during repair 
(CASSANDRA-9249)
   * Disable single sstable tombstone compactions for DTCS by default 
(CASSANDRA-9234)
 - * Do more agressive ttl expiration checks to be able to
 -   drop more sstables (CASSANDRA-8243)
   * IncomingTcpConnection thread is not named (CASSANDRA-9262)
   * Close incoming connections when MessagingService is stopped 
(CASSANDRA-9238)
 +
- 
 +2.1.5
 + * Re-add deprecated cold_reads_to_omit param for backwards compat 
(CASSANDRA-9203)
 + * Make anticompaction visible in compactionstats (CASSANDRA-9098)
 + * Improve nodetool getendpoints documentation about the partition
 +   key parameter (CASSANDRA-6458)
 + * Don't check other keyspaces for schema changes when an user-defined
 +   type is altered (CASSANDRA-9187)
 + * Allow takeColumnFamilySnapshot to take a list of tables (CASSANDRA-8348)
 + * Limit major sstable operations to their canonical representation 
(CASSANDRA-8669)
 + * cqlsh: Add tests for INSERT and UPDATE tab completion (CASSANDRA-9125)
 + * cqlsh: quote column names when needed in COPY FROM inserts (CASSANDRA-9080)
 + * Add generate-idea-files target to build.xml (CASSANDRA-9123)
 + * Do not load read meter for offline operations (CASSANDRA-9082)
 + * cqlsh: Make CompositeType data readable (CASSANDRA-8919)
 + * cqlsh: Fix display of triggers (CASSANDRA-9081)
 + * Fix NullPointerException when deleting or setting an element by index on
 +   a null list collection (CASSANDRA-9077)
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation

[jira] [Updated] (CASSANDRA-9269) Huge commitlog not flushed.

2015-04-30 Thread Moloud Shahbazi (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Moloud Shahbazi updated CASSANDRA-9269:
---
Priority: Trivial  (was: Major)

 Huge commitlog not flushed.
 ---

 Key: CASSANDRA-9269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9269
 Project: Cassandra
  Issue Type: Wish
 Environment: ubuntu
 cassandra-2.1.3
Reporter: Moloud Shahbazi
Priority: Trivial

 I wrote at once a lot of data in several column families of keyspaceName on a 
 single node cluster. I only read from this keyspace afterward. My issue is 
 that size of commitlog is huge and does not shrink:
 $ du -sh data/*
 7.8Gdata/commitlog
 7.0Gdata/data
 36M data/saved_caches
 When I try to flush using nodetool, if i run ./bin/nodetool flush no error 
 occurs and no change happens in size of commit log.  even when I specify 
 keyspaceName no change happens.
 Also Fot further information, the following is description of my keyspace 
 schema, 
 {code}
 cqlsh describe keyspace keyspaceName;
 CREATE KEYSPACE keyspaceName WITH replication = {'class': 
 'NetworkTopologyStrategy', 'datacenter1': '1'}  AND durable_writes = true;
 CREATE TABLE keyspaceName.CF0 (
 concept_id text,
 chunk_id int,
 json_data text,
 PRIMARY KEY (concept_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF1 (
 item_id text,
 chunk_id int,
 reviews text,
 PRIMARY KEY (item_id, chunk_id)
 ) WITH CLUSTERING ORDER BY (chunk_id ASC)
 AND bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 CREATE TABLE keyspaceName.CF2 (
 review_id text PRIMARY KEY,
 conceptids text,
 score int
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.0
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9270) Running resetlocalschema during repair can cause repair to hang

T Jake Luciani created CASSANDRA-9270:
-

 Summary: Running resetlocalschema during repair can cause repair 
to hang
 Key: CASSANDRA-9270
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9270
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Assignee: Yuki Morishita
Priority: Minor
 Fix For: 2.0.x


If you run resetlocalschema during a repair the node doing the repair can hang.

The following test reproduces the issue quite frequently:
https://github.com/riptano/cassandra-dtest/pull/247

this is from trunk, but happens in 2.0 and 2.1 as well
{code}
ERROR [Repair#1:10] 2015-04-30 12:57:56,675 CassandraDaemon.java: Exception in 
thread Thread[Repair#1:10,5,RMI Runtime]
java.lang.IllegalArgumentException: Unknown keyspace/cf pair 
(keyspace1.standard1)
at 
org.apache.cassandra.db.Keyspace.getColumnFamilyStore(Keyspace.java:172) 
~[main/:na]
at 
org.apache.cassandra.repair.RepairJob.sendValidationRequest(RepairJob.java:189) 
~[main/:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:110) 
~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_72]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
~[na:1.7.0_72]
at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_72]

{code}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7486) Compare CMS and G1 pause times


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521805#comment-14521805
 ] 

Ariel Weisberg edited comment on CASSANDRA-7486 at 4/30/15 4:49 PM:


I would just like to see the data visualized. If it's not better in every 
dimension then in what dimensions is better/worse across all the data that Al 
collected.


was (Author: aweisberg):
I would just like to see the data visualized. If it's not better in every 
dimension then in what dimensions is better/worse in across all the data that 
Al collected.

 Compare CMS and G1 pause times
 --

 Key: CASSANDRA-7486
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
 Project: Cassandra
  Issue Type: Test
  Components: Config
Reporter: Jonathan Ellis
Assignee: Shawn Kumar
 Fix For: 2.1.x


 See 
 http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
  and https://twitter.com/rbranson/status/482113561431265281
 May want to default 2.1 to G1.
 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
 Suspect this will help G1 even more than CMS.  (NB this is off by default but 
 needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9270) Running resetlocalschema during repair can cause repair to hang


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-9270:
--
Description: 
If you run resetlocalschema during a repair the node doing the repair can hang.

The following test reproduces the issue quite frequently:
https://github.com/riptano/cassandra-dtest/pull/247

This is from trunk, but happens in 2.0 and 2.1 as well.  My guess is there is 
some count down latch that isn't count down when the repair msg fails to be 
parsed.

{code}
ERROR [Repair#1:10] 2015-04-30 12:57:56,675 CassandraDaemon.java: Exception in 
thread Thread[Repair#1:10,5,RMI Runtime]
java.lang.IllegalArgumentException: Unknown keyspace/cf pair 
(keyspace1.standard1)
at 
org.apache.cassandra.db.Keyspace.getColumnFamilyStore(Keyspace.java:172) 
~[main/:na]
at 
org.apache.cassandra.repair.RepairJob.sendValidationRequest(RepairJob.java:189) 
~[main/:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:110) 
~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_72]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
~[na:1.7.0_72]
at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_72]

{code}





  was:
If you run resetlocalschema during a repair the node doing the repair can hang.

The following test reproduces the issue quite frequently:
https://github.com/riptano/cassandra-dtest/pull/247

this is from trunk, but happens in 2.0 and 2.1 as well
{code}
ERROR [Repair#1:10] 2015-04-30 12:57:56,675 CassandraDaemon.java: Exception in 
thread Thread[Repair#1:10,5,RMI Runtime]
java.lang.IllegalArgumentException: Unknown keyspace/cf pair 
(keyspace1.standard1)
at 
org.apache.cassandra.db.Keyspace.getColumnFamilyStore(Keyspace.java:172) 
~[main/:na]
at 
org.apache.cassandra.repair.RepairJob.sendValidationRequest(RepairJob.java:189) 
~[main/:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:110) 
~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_72]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
~[na:1.7.0_72]
at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_72]

{code}




 Running resetlocalschema during repair can cause repair to hang
 ---

 Key: CASSANDRA-9270
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9270
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Assignee: Yuki Morishita
Priority: Minor
 Fix For: 2.0.x


 If you run resetlocalschema during a repair the node doing the repair can 
 hang.
 The following test reproduces the issue quite frequently:
 https://github.com/riptano/cassandra-dtest/pull/247
 This is from trunk, but happens in 2.0 and 2.1 as well.  My guess is there is 
 some count down latch that isn't count down when the repair msg fails to be 
 parsed.
 {code}
 ERROR [Repair#1:10] 2015-04-30 12:57:56,675 CassandraDaemon.java: Exception 
 in thread Thread[Repair#1:10,5,RMI Runtime]
 java.lang.IllegalArgumentException: Unknown keyspace/cf pair 
 (keyspace1.standard1)
   at 
 org.apache.cassandra.db.Keyspace.getColumnFamilyStore(Keyspace.java:172) 
 ~[main/:na]
   at 
 org.apache.cassandra.repair.RepairJob.sendValidationRequest(RepairJob.java:189)
  ~[main/:na]
   at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:110) 
 ~[main/:na]
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_72]
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  ~[na:1.7.0_72]
   at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_72]
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9140) Scrub should handle corrupted compressed chunks


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-9140:
---
Attachment: 9140-trunk.txt
9140-2.1.txt
9140-2.0.txt

 Scrub should handle corrupted compressed chunks
 ---

 Key: CASSANDRA-9140
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9140
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Stefania
 Fix For: 2.0.15, 2.1.6

 Attachments: 9140-2.0.txt, 9140-2.1.txt, 9140-trunk.txt


 Scrub can handle corruption within a row, but can't handle corruption of a 
 compressed sstable that results in being unable to decompress a chunk.  Since 
 the majority of Cassandra users are probably running with compression 
 enabled, it's important that scrub be able to handle this (likely more 
 common) form of sstable corruption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9151) Anti-compaction is blocking ANTI_ENTROPY stage

2015-04-30 Thread sankalp kohli (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522067#comment-14522067
]

sankalp kohli commented on CASSANDRA-9151:
--

I am +1 on the patch which I applied to 2.1. While looking at the 2.1 branch, I
found one more potential issue not related to the patch.
We could potentially block this stage again since many methods are synchronized
in ActiveRepairService.
Methods prepareForRepair(could block for 1 hour for prepare message response)
and finishParentSession(this one block for anticompaction to finish) are
synchronized and could hold on to the lock for a long time.
In RepairMessageVerbHandler.doVerb, if there is an exception for another
repair, removeParentRepairSession(also synchronized) will block.

I am not sure whether we can take this in another JIRA as this is blocking lot
of people.

Anti-compaction is blocking ANTI_ENTROPY stage
---

Key: CASSANDRA-9151
URL: https://issues.apache.org/jira/browse/CASSANDRA-9151
Project: Cassandra
Issue Type: Bug
Reporter: sankalp kohli
Assignee: Yuki Morishita
Fix For: 3.x, 2.1.x

Attachments:
0001-Do-not-block-stage-when-performing-anticompaction.patch

In RepairMessageVerbHandler.doVerb case ANTICOMPACTION_REQUEST
We block on ANTI_ENTROPY stage. This is not good as no other message could be
processed till any anticompaction is running. This can cause snapshots for
new repairs to fail.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6477) Global indexes

2015-04-30 Thread Matthias Broecheler (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522136#comment-14522136
]

Matthias Broecheler commented on CASSANDRA-6477:

I think the discussion around materialized views (which I would love to see in
C* at some point) is distracting from what this ticket is really about: closing
a hole in the indexing story for C*.

In RDBMS (and pretty much all other database systems), indexes are used to
efficiently retrieve a set of rows identified by their columns values in a
particular order at the expense of write performance. By design, C* builds a
100% selectivity index on the primary key. In addition, one can install
secondary indexes. Those secondary indexes are useful up to a certain
selectivity %. Beyond that threshold, it becomes increasingly more efficient to
maintain the index as a global distributed hash map rather than a local index
on each node. And that's the hole in the indexing story, because those types of
indexes must currently be maintained by the application.

I am stating the obvious here to point out that the first problem is to provide
the infrastructure to create that second class of indexes while ensuring some
form of (eventual) consistency. Much like with 2i, once that is in place one
can utilize the infrastructure to build other things on top - including
materialized views which will need this to begin with (if the primary key of
your materialized view has high selectivity).

As for nomenclature, I agree that global vs local index is a technical
distinction that has little to no meaning to the user. After all, they want to
build an index to get to their data quickly. How that happens is highly
secondary. Initially, it might make sense to ask the user to specify the
selectivity estimate for the index (defaulting to low) and for C* to pick the
best indexing approach based on that. In the future, one could utilize sampled
histograms to help the user with that decision.

Global indexes
--

Key: CASSANDRA-6477
URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
Labels: cql
Fix For: 3.x

Local indexes are suitable for low-cardinality data, where spreading the
index across the cluster is a Good Thing. However, for high-cardinality
data, local indexes require querying most nodes in the cluster even if only a
handful of rows is returned.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9166) Prepared statements using functions in collection literals aren't invalidated when functions are dropped


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-9166:
-
Reviewer: Aleksey Yeschenko

 Prepared statements using functions in collection literals aren't invalidated 
 when functions are dropped
 

 Key: CASSANDRA-9166
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9166
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
  Labels: cql, functions
 Fix For: 3.x


 When a function is dropped, any prepared statements which reference it need 
 to be removed from the prepared statement cache. 
 The default implementation of {{Term#usesFunction}} in {{Term.NonTerminal}} 
 is not overriden in all the places it should be. The {{DelayedValue}} classes 
 in {{Lists}}, {{Sets}}, {{Maps}} and {{Tuples}} may all make use of function 
 calls.
 {code}
 CREATE KEYSPACE ks WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 1};
 CREATE TABLE ks.t1 (k int PRIMARY KEY, v listint);
 CREATE FUNCTION ks.echo_int(input int) RETURNS int LANGUAGE javascript AS 
 'input';
 {code}
 a prepared statement of the form:
 {code}
 INSERT INTO ks.t1 (k, v) VALUES (?, [ks.echo_int(?)]);
 {code}
 should be dropped when {{ks.echo_int(int)}} is, but currently that isn't the 
 case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9273) Avoid calculating maxPurgableTimestamp for partitions containing non-expired TTL cells


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-9273:
---
Summary: Avoid calculating maxPurgableTimestamp for partitions containing 
non-expired TTL cells  (was: Avoid calculating maxPurgableTimestamp for 
partitions with non-expired TTL)

 Avoid calculating maxPurgableTimestamp for partitions containing non-expired 
 TTL cells
 --

 Key: CASSANDRA-9273
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9273
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
 Fix For: 2.1.x


 Seems we still calculate maxPurgableTimestamp for partitions where we have 
 TTL that is known to not be expired, we should try to avoid that



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9166) Prepared statements using functions in collection literals aren't invalidated when functions are dropped

2015-04-30 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522047#comment-14522047
 ] 

Sam Tunnicliffe commented on CASSANDRA-9166:


Pushed a branch with a fix [here|https://github.com/beobal/cassandra/tree/9166]
The first commit just adds a utest and a minimal fix. The second commit removes 
the {{useFunction}} method from the various classes  interfaces in the cql3 
package that have it. Instead, we can re-use {{getFunctions}} to check whether 
a prepared statement references a function when it gets dropped. The third 
commit just fixes a bug caught by {{AggregationTest}} following the removal of 
{{usesFunction}}.

 Prepared statements using functions in collection literals aren't invalidated 
 when functions are dropped
 

 Key: CASSANDRA-9166
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9166
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
  Labels: cql, functions
 Fix For: 3.x


 When a function is dropped, any prepared statements which reference it need 
 to be removed from the prepared statement cache. 
 The default implementation of {{Term#usesFunction}} in {{Term.NonTerminal}} 
 is not overriden in all the places it should be. The {{DelayedValue}} classes 
 in {{Lists}}, {{Sets}}, {{Maps}} and {{Tuples}} may all make use of function 
 calls.
 {code}
 CREATE KEYSPACE ks WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 1};
 CREATE TABLE ks.t1 (k int PRIMARY KEY, v listint);
 CREATE FUNCTION ks.echo_int(input int) RETURNS int LANGUAGE javascript AS 
 'input';
 {code}
 a prepared statement of the form:
 {code}
 INSERT INTO ks.t1 (k, v) VALUES (?, [ks.echo_int(?)]);
 {code}
 should be dropped when {{ks.echo_int(int)}} is, but currently that isn't the 
 case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9190) Map keys aren’t properly serialized as strings in SELECT JSON queries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-9190:
---
Fix Version/s: 3.0

 Map keys aren’t properly serialized as strings in SELECT JSON queries
 -

 Key: CASSANDRA-9190
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9190
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Gianluca Righetto
Assignee: Tyler Hobbs
 Fix For: 3.0


 When I run a SELECT JSON query on a table that contains a column of type map, 
 the JSON output doesn’t wrap the map keys with quotes, thus creating a 
 non-standard JSON representation, which doesn’t play along with JSON parsers 
 very well. 
 Here’s an example where the map keys are actually a set:
 {code}
 create table table1 (id int primary key, mymap mapfrozensetint,int);
 insert into table1 (id,mymap) values (1, {{1,2,3,4}:1});
 select json id,mymap from table1;
 {code}
 The output is:
 {noformat}
 {id: 1, mymap: {[1, 2, 3, 4]: 1}}
 {noformat}
 But what I expected was that the map key, in this case the entire integer 
 set, would be wrapped with quotes.
 The same thing happens when the key is a primitive, such as int, or when it’s 
 another type of collection.
 I tried this with the latest version of the Cassandra codebase from the 
 repository’s trunk (1f65a12c33).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9272) Atomic Batch Statements Fail to Execute on Some Coordinators

2015-04-30 Thread Jared Gray (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jared Gray updated CASSANDRA-9272:
--
Environment: 
Operating System: Red Hat Enterprise Linux Server release 6.5 (Santiago) |
Cassandra and CQL: [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift 
protocol 19.39.0] |
JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_79 (Oracle Java) |
Using PropertyFileSnitch to define cluster topology.

  was:
Operating System: Red Hat Enterprise Linux Server release 6.5 (Santiago)
Cassandra and CQL: [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift 
protocol 19.39.0]
JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_79 (Oracle Java)
Using PropertyFileSnitch to define cluster topology.


 Atomic Batch Statements Fail to Execute on Some Coordinators
 

 Key: CASSANDRA-9272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9272
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Operating System: Red Hat Enterprise Linux Server 
 release 6.5 (Santiago) |
 Cassandra and CQL: [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift 
 protocol 19.39.0] |
 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_79 (Oracle Java) |
 Using PropertyFileSnitch to define cluster topology.
Reporter: Jared Gray

 Install and configure a three-node Cassandra cluster spread across two data 
 centers (DCs). Place two Cassandra instances in DC1 and one Cassandra 
 instance in DC2. (We have designated one node as a seed in each data center.)
 Gracefully shut down any one Cassandra instance in DC1. This can be done by 
 running 'kill process_id' in Linux.
 Open a CQL Shell connection to the remaining node in DC1.
 Create the following keyspace: CREATE KEYSPACE batch_test WITH replication = 
 {'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};
 Create the following table: CREATE TABLE batch_test.batch_table ( 
 partition_key TEXT PRIMARY KEY );
 Execute the following atomic batch query: BEGIN BATCH INSERT INTO 
 batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
 batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH;
 Observe an availability-related error, despite a sufficient number of nodes 
 available to service the query. The trace below shows sample results from our 
 cluster:
 DC1
 
 cqlsh CONSISTENCY
 Current consistency level is ONE.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 Unable to complete request: one or more nodes were unavailable.
 cqlsh CONSISTENCY QUORUM
 Consistency level set to QUORUM.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 Unable to complete request: one or more nodes were unavailable.
 cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('a'); INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('b'); APPLY BATCH;
 cqlsh
 Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
 successfully if it only contains one INSERT query.
 We now show a similar trace using a CQL Shell connected to the Cassandra 
 instance in DC2:
 DC2
 ---
 cqlsh CONSISTENCY
 Current consistency level is ONE.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 cqlsh CONSISTENCY QUORUM
 Consistency level set to QUORUM.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('a'); INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('b'); APPLY BATCH;
 cqlsh
 All queries execute successfully using the DC2 Cassandra instance as a 
 coordinator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9275) ReadMessageTest.testNoCommitLog fails with test-compression

Ariel Weisberg created CASSANDRA-9275:
-

 Summary: ReadMessageTest.testNoCommitLog fails with 
test-compression
 Key: CASSANDRA-9275
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9275
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Branimir Lambov


It's reading the raw bytes of the commit log expecting to find an uncompressed 
log and instead finding a compressed log and now the test is not working if the 
log is compressed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9276) StreamingTransferTest fails under test-compression due to bad assertion

Ariel Weisberg created CASSANDRA-9276:
-

 Summary: StreamingTransferTest fails under test-compression due to 
bad assertion
 Key: CASSANDRA-9276
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9276
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg


https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
{noformat}
assert in.getBytesRead()  totalSize;
{noformat]
My guess is that total size is the compressed size, not the uncompressed size. 
Remove the assertion and the test passes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6477) Global indexes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522182#comment-14522182
 ] 

Aleksey Yeschenko commented on CASSANDRA-6477:
--

bq. Do you mean that from an implementation perspective?

Yes.

 Global indexes
 --

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
  Labels: cql
 Fix For: 3.x


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6702) Upgrading node uses the wrong port in gossiping

2015-04-30 Thread Blake Eggleston (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522129#comment-14522129
 ] 

Blake Eggleston commented on CASSANDRA-6702:


Same problem upgrading from 1.1.x to 2.0.x.

The crux of the issue is here: 
https://github.com/apache/cassandra/blob/cassandra-1.2/src/java/org/apache/cassandra/net/IncomingTcpConnection.java#L129

When an upgraded node connects to a non-upgraded node, that call to 
Gossiper.instance.addSavedNode marks the connecting node as down and forces it 
to reconnect. That's usually not a problem, except when using the 
Ec2MultiRegionSnitch, since the ReconnectableSnitchHelper reconnects 
onJoin/onChange/onAlive, which starts a cycle of reconnection. So this was 
worked around in CASSANDRA-5669 by just skipping the reconnect step if the 
cluster was upgrading here: 
https://github.com/apache/cassandra/blob/cassandra-1.2/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java#L65

However for upgrades from versions = 1.2 to versions = 1.2, I think we should 
be able to skip that gossip call, since MS protocol version is no longer 
tracked by gossip as of 1.2. If the call was just making sure we reconnected, 
we should be able to replace it with a call to 
MessagingService.instance.getConnectionPool for the same effect.

With that out of the way, we should be able to remove the version check from 
ReconnectableSnitchHelper. That should probably go behind a -D flag until 3.0 
though, since all hell will break loose if someone using the 
Ec2MultiRegionSnitch upgrades from a release without that fix

 Upgrading node uses the wrong port in gossiping
 ---

 Key: CASSANDRA-6702
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6702
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 1.1.7, AWS, Ec2MultiRegionSnitch
Reporter: Minh Do
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.0.x


 When upgrading a node in 1.1.7 (or 1.1.11) cluster to 1.2.15 and inspecting 
 the gossip information on port/Ip, I could see that the upgrading node (1.2 
 version) communicates to one other node in the same region using Public IP 
 and non-encrypted port.
 For the rest, the upgrading node uses the correct ports and IPs to 
 communicate in this manner:
Same region: private IP and non-encrypted port 
and
Different region: public IP and encrypted port
 Because there is one node like this (or 2 out of 12 nodes cluster in which 
 nodes are split equally on 2 AWS regions), we have to modify Security Group 
 to allow the new traffics.
 Without modifying the SG, the 95th and 99th latencies for both reads and 
 writes in the cluster are very bad (due to RPC timeout).  Inspecting closer, 
 that upgraded node (1.2 node) is contributing to all of the high latencies 
 whenever it acts as a coordinator node. 
  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9272) Atomic Batch Statements Fail to Execute on Some Coordinators

2015-04-30 Thread Jared Gray (JIRA)

Jared Gray created CASSANDRA-9272:
-

 Summary: Atomic Batch Statements Fail to Execute on Some 
Coordinators
 Key: CASSANDRA-9272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9272
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Operating System: Red Hat Enterprise Linux Server release 
6.5 (Santiago)
Cassandra and CQL: [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift 
protocol 19.39.0]
JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_79 (Oracle Java)
Using PropertyFileSnitch to define cluster topology.
Reporter: Jared Gray


Install and configure a three-node Cassandra cluster spread across two data 
centers (DCs). Place two Cassandra instances in DC1 and one Cassandra instance 
in DC2. (We have designated one node as a seed in each data center.)

Gracefully shut down any one Cassandra instance in DC1. This can be done by 
running 'kill process_id' in Linux.

Open a CQL Shell connection to the remaining node in DC1.

Create the following keyspace: CREATE KEYSPACE batch_test WITH replication = 
{'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};

Create the following table: CREATE TABLE batch_test.batch_table ( partition_key 
TEXT PRIMARY KEY );

Execute the following atomic batch query: BEGIN BATCH INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH;

Observe an availability-related error, despite a sufficient number of nodes 
available to service the query. The trace below shows sample results from our 
cluster:

DC1

cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh

Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
successfully if it only contains one INSERT query.

We now show a similar trace using a CQL Shell connected to the Cassandra 
instance in DC2:

DC2
---
cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh

All queries execute successfully using the DC2 Cassandra instance as a 
coordinator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9266) Repair failed with error Already repairing SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'), can not continue.

2015-04-30 Thread Razi Khaja (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522062#comment-14522062
 ] 

Razi Khaja commented on CASSANDRA-9266:
---

I've attached a partial system log file (I couldnt attach the whole log file, 
it is around 13MB and the limit is 10MB here in JIRA), around the time that 
incremental parallel repair failed, including any ERROR,WARN or Exception 
messages, including 10 lines before and after.  
{code}
egrep -i 2015-04-29 13:4|ERROR|WARN|Exception -B10 -A10 
cassandra_system.log.CASSANDRA-9266.txt  
cassandra_system.log.partial.CASSANDRA-9266.txt
{code}

 Repair failed with error Already repairing 
 SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'),
  can not continue.
 ---

 Key: CASSANDRA-9266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9266
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.3
Reporter: Razi Khaja
 Fix For: 2.1.x

 Attachments: cassandra_system.log.partial.CASSANDRA-9266.txt


 I am running incremental parallel repair using the following command:
 {code}
 nodetool --host my_hostname repair --incremental --in-local-dc --parallel
 {code}
 and get the following error:
 {code}
 Repair failed with error Already repairing 
 SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'),
  can not continue.
 {code}
 I have 3 data centers, each with 4 nodes. Neither incremental or full repair 
 is running on any of my other nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9272) Atomic Batch Statements Fail to Execute on Some Coordinators


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9272:
---
Reproduced In: 2.0.14, 2.0.6  (was: 2.0.6, 2.0.14)
Fix Version/s: 2.0.x

 Atomic Batch Statements Fail to Execute on Some Coordinators
 

 Key: CASSANDRA-9272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9272
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Operating System: Red Hat Enterprise Linux Server 
 release 6.5 (Santiago) |
 Cassandra and CQL: [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift 
 protocol 19.39.0] |
 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_79 (Oracle Java) |
 Using PropertyFileSnitch to define cluster topology.
Reporter: Jared Gray
 Fix For: 2.0.x


 Install and configure a three-node Cassandra cluster spread across two data 
 centers (DCs). Place two Cassandra instances in DC1 and one Cassandra 
 instance in DC2. (We have designated one node as a seed in each data center.)
 Gracefully shut down any one Cassandra instance in DC1. This can be done by 
 running 'kill process_id' in Linux.
 Open a CQL Shell connection to the remaining node in DC1.
 Create the following keyspace: CREATE KEYSPACE batch_test WITH replication = 
 {'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};
 Create the following table: CREATE TABLE batch_test.batch_table ( 
 partition_key TEXT PRIMARY KEY );
 Execute the following atomic batch query: BEGIN BATCH INSERT INTO 
 batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
 batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH;
 Observe an availability-related error, despite a sufficient number of nodes 
 available to service the query. The trace below shows sample results from our 
 cluster:
 DC1
 
 cqlsh CONSISTENCY
 Current consistency level is ONE.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 Unable to complete request: one or more nodes were unavailable.
 cqlsh CONSISTENCY QUORUM
 Consistency level set to QUORUM.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 Unable to complete request: one or more nodes were unavailable.
 cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('a'); INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('b'); APPLY BATCH;
 cqlsh
 Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
 successfully if it only contains one INSERT query.
 We now show a similar trace using a CQL Shell connected to the Cassandra 
 instance in DC2:
 DC2
 ---
 cqlsh CONSISTENCY
 Current consistency level is ONE.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 cqlsh CONSISTENCY QUORUM
 Consistency level set to QUORUM.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('a'); INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('b'); APPLY BATCH;
 cqlsh
 All queries execute successfully using the DC2 Cassandra instance as a 
 coordinator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9274) Changing memtable_flush_writes per recommendations in cassandra.yaml causes memtable_cleanup_threshold to be too small

2015-04-30 Thread Donald Smith (JIRA)

Donald Smith created CASSANDRA-9274:
---

 Summary: Changing memtable_flush_writes per recommendations in 
cassandra.yaml causes  memtable_cleanup_threshold to be too small
 Key: CASSANDRA-9274
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9274
 Project: Cassandra
  Issue Type: Improvement
Reporter: Donald Smith
Priority: Minor


It says in cassandra.yaml:
{noformat}
# If your data directories are backed by SSD, you should increase this
# to the number of cores.
#memtable_flush_writers: 8
{noformat}
so we raised it to 24.

Much later we noticed a warning in the logs:
{noformat}
WARN  [main] 2015-04-22 15:32:58,619 DatabaseDescriptor.java:539 - 
memtable_cleanup_threshold is set very low, which may cause performance 
degradation
{noformat}
Looking at cassandra.yaml again I see:
{noformat}
# memtable_cleanup_threshold defaults to 1 / (memtable_flush_writers + 1)
# memtable_cleanup_threshold: 0.11
#memtable_cleanup_threshold: 0.11
{noformat}
So, I uncommented that last line (figuring that 0.11 is a reasonable value).

Cassandra.yaml should give better guidance or the code should *prevent* the 
value from going outside a reasonable range.
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6477) Global indexes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522140#comment-14522140
 ] 

Aleksey Yeschenko commented on CASSANDRA-6477:
--

bq. Much like with 2i, once that is in place one can utilize the infrastructure 
to build other things on top - including materialized views which will need 
this to begin with (if the primary key of your materialized view has high 
selectivity).

Sorry for a short reply. Just wanted to note that I see this being the other 
way around. Full-on materialized views are the infrastructure for 'global 
indexes', not the other way around.

 Global indexes
 --

 Key: CASSANDRA-6477
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
  Labels: cql
 Fix For: 3.x


 Local indexes are suitable for low-cardinality data, where spreading the 
 index across the cluster is a Good Thing.  However, for high-cardinality 
 data, local indexes require querying most nodes in the cluster even if only a 
 handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-6702) Upgrading node uses the wrong port in gossiping

2015-04-30 Thread Blake Eggleston (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston reassigned CASSANDRA-6702:
--

Assignee: Blake Eggleston  (was: Brandon Williams)

 Upgrading node uses the wrong port in gossiping
 ---

 Key: CASSANDRA-6702
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6702
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 1.1.7, AWS, Ec2MultiRegionSnitch
Reporter: Minh Do
Assignee: Blake Eggleston
Priority: Minor
 Fix For: 2.0.x


 When upgrading a node in 1.1.7 (or 1.1.11) cluster to 1.2.15 and inspecting 
 the gossip information on port/Ip, I could see that the upgrading node (1.2 
 version) communicates to one other node in the same region using Public IP 
 and non-encrypted port.
 For the rest, the upgrading node uses the correct ports and IPs to 
 communicate in this manner:
Same region: private IP and non-encrypted port 
and
Different region: public IP and encrypted port
 Because there is one node like this (or 2 out of 12 nodes cluster in which 
 nodes are split equally on 2 AWS regions), we have to modify Security Group 
 to allow the new traffics.
 Without modifying the SG, the 95th and 99th latencies for both reads and 
 writes in the cluster are very bad (due to RPC timeout).  Inspecting closer, 
 that upgraded node (1.2 node) is contributing to all of the high latencies 
 whenever it acts as a coordinator node. 
  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9276) StreamingTransferTest fails under test-compression due to bad assertion


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9276:
--
Description: 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
{noformat}
assert in.getBytesRead()  totalSize;
{noformat}
My guess is that total size is the compressed size, not the uncompressed size. 
Remove the assertion and the test passes.

Total size is calculated with
{noformat}
long size = 0;
// calculate total length of transferring chunks
for (CompressionMetadata.Chunk chunk : compressionInfo.chunks)
size += chunk.length + 4; // 4 bytes for CRC
return size;
{noformat}

  was:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
{noformat}
assert in.getBytesRead()  totalSize;
{noformat}
My guess is that total size is the compressed size, not the uncompressed size. 
Remove the assertion and the test passes.


 StreamingTransferTest fails under test-compression due to bad assertion
 ---

 Key: CASSANDRA-9276
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9276
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.x


 https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
 {noformat}
 assert in.getBytesRead()  totalSize;
 {noformat}
 My guess is that total size is the compressed size, not the uncompressed 
 size. Remove the assertion and the test passes.
 Total size is calculated with
 {noformat}
 long size = 0;
 // calculate total length of transferring chunks
 for (CompressionMetadata.Chunk chunk : compressionInfo.chunks)
 size += chunk.length + 4; // 4 bytes for CRC
 return size;
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9266) Repair failed with error Already repairing SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'), can not continue.

2015-04-30 Thread Razi Khaja (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Razi Khaja updated CASSANDRA-9266:
--
Attachment: cassandra_system.log.partial.CASSANDRA-9266.txt

 Repair failed with error Already repairing 
 SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'),
  can not continue.
 ---

 Key: CASSANDRA-9266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9266
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1.3
Reporter: Razi Khaja
 Fix For: 2.1.x

 Attachments: cassandra_system.log.partial.CASSANDRA-9266.txt


 I am running incremental parallel repair using the following command:
 {code}
 nodetool --host my_hostname repair --incremental --in-local-dc --parallel
 {code}
 and get the following error:
 {code}
 Repair failed with error Already repairing 
 SSTableReader(path='/path/to/keyspace/column_family/keyspace-column_family--Data.db'),
  can not continue.
 {code}
 I have 3 data centers, each with 4 nodes. Neither incremental or full repair 
 is running on any of my other nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9272) Atomic Batch Statements Fail to Execute on Some Coordinators


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-9272:
---
Description: 
Install and configure a three-node Cassandra cluster spread across two data 
centers (DCs). Place two Cassandra instances in DC1 and one Cassandra instance 
in DC2. (We have designated one node as a seed in each data center.)

Gracefully shut down any one Cassandra instance in DC1. This can be done by 
running 'kill process_id' in Linux.

Open a CQL Shell connection to the remaining node in DC1.

Create the following keyspace: {{CREATE KEYSPACE batch_test WITH replication = 
\{'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'\};}}

Create the following table: {{CREATE TABLE batch_test.batch_table ( 
partition_key TEXT PRIMARY KEY );}}

Execute the following atomic batch query: {{BEGIN BATCH INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH}};

Observe an availability-related error, despite a sufficient number of nodes 
available to service the query. The trace below shows sample results from our 
cluster:

DC1

{noformat}
cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh
{noformat}

Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
successfully if it only contains one INSERT query.

We now show a similar trace using a CQL Shell connected to the Cassandra 
instance in DC2:

DC2
---
{noformat}
cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh
{noformat}

All queries execute successfully using the DC2 Cassandra instance as a 
coordinator.

  was:
Install and configure a three-node Cassandra cluster spread across two data 
centers (DCs). Place two Cassandra instances in DC1 and one Cassandra instance 
in DC2. (We have designated one node as a seed in each data center.)

Gracefully shut down any one Cassandra instance in DC1. This can be done by 
running 'kill process_id' in Linux.

Open a CQL Shell connection to the remaining node in DC1.

Create the following keyspace: {{CREATE KEYSPACE batch_test WITH replication = 
{'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};}}

Create the following table: {{CREATE TABLE batch_test.batch_table ( 
partition_key TEXT PRIMARY KEY );}}

Execute the following atomic batch query: {{BEGIN BATCH INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH}};

Observe an availability-related error, despite a sufficient number of nodes 
available to service the query. The trace below shows sample results from our 
cluster:

DC1

{noformat}
cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh
{noformat}

Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
successfully if it only contains one INSERT query.

We now show a similar trace using a CQL Shell connected to the Cassandra 
instance in DC2:

DC2

[jira] [Updated] (CASSANDRA-9272) Atomic Batch Statements Fail to Execute on Some Coordinators


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-9272:
---
Description: 
Install and configure a three-node Cassandra cluster spread across two data 
centers (DCs). Place two Cassandra instances in DC1 and one Cassandra instance 
in DC2. (We have designated one node as a seed in each data center.)

Gracefully shut down any one Cassandra instance in DC1. This can be done by 
running 'kill process_id' in Linux.

Open a CQL Shell connection to the remaining node in DC1.

Create the following keyspace: {{CREATE KEYSPACE batch_test WITH replication = 
{'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};}}

Create the following table: {{CREATE TABLE batch_test.batch_table ( 
partition_key TEXT PRIMARY KEY );}}

Execute the following atomic batch query: {{BEGIN BATCH INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH}};

Observe an availability-related error, despite a sufficient number of nodes 
available to service the query. The trace below shows sample results from our 
cluster:

DC1

{noformat}
cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh
{noformat}

Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
successfully if it only contains one INSERT query.

We now show a similar trace using a CQL Shell connected to the Cassandra 
instance in DC2:

DC2
---
{noformat}
cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh
{noformat}

All queries execute successfully using the DC2 Cassandra instance as a 
coordinator.

  was:
Install and configure a three-node Cassandra cluster spread across two data 
centers (DCs). Place two Cassandra instances in DC1 and one Cassandra instance 
in DC2. (We have designated one node as a seed in each data center.)

Gracefully shut down any one Cassandra instance in DC1. This can be done by 
running 'kill process_id' in Linux.

Open a CQL Shell connection to the remaining node in DC1.

Create the following keyspace: CREATE KEYSPACE batch_test WITH replication = 
{'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};

Create the following table: CREATE TABLE batch_test.batch_table ( partition_key 
TEXT PRIMARY KEY );

Execute the following atomic batch query: BEGIN BATCH INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH;

Observe an availability-related error, despite a sufficient number of nodes 
available to service the query. The trace below shows sample results from our 
cluster:

DC1

cqlsh CONSISTENCY
Current consistency level is ONE.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh CONSISTENCY QUORUM
Consistency level set to QUORUM.
cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
BATCH;
Unable to complete request: one or more nodes were unavailable.
cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table (partition_key) 
VALUES ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); 
APPLY BATCH;
cqlsh

Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
successfully if it only contains one INSERT query.

We now show a similar trace using a CQL Shell connected to the Cassandra 
instance in DC2:

DC2
---
cqlsh CONSISTENCY
Current

[jira] [Updated] (CASSANDRA-9061) Add backoff and recovery to cqlsh COPY FROM when write timeouts occur

[
https://issues.apache.org/jira/browse/CASSANDRA-9061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tyler Hobbs updated CASSANDRA-9061:
---
Attachment: 9061-suggested.txt

I think we can take a somewhat simpler approach. I've attached a patch that is
untested but demonstrates roughly what I'm thinking of. (Most of the diff is
just an indentation change.)

I don't think we really need to track in-progress or successful operations. We
can rely on the connection's {{in_flight}} count to know if there are
in-progress operations. Successful operations don't require any further
action. We don't need to track the request ID, because it's automatically
released by the connection when it gets a response, and we can easily get a new
one. Am I missing something in my suggested approach?

I think we're going to need a dtest to exercise this. You could start from
{{cqlsh_tests.TestCqlsh.test_copy_to()}}, setting a low enough write timeout
that a large number of operations fail.

Add backoff and recovery to cqlsh COPY FROM when write timeouts occur
-

Key: CASSANDRA-9061
URL: https://issues.apache.org/jira/browse/CASSANDRA-9061
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Assignee: Carl Yeksigian
Priority: Minor
Labels: cqlsh
Fix For: 2.1.x

Attachments: 9061-2.1.txt, 9061-suggested.txt

Previous versions of COPY FROM didn't handle write timeouts because it was
rarely fast enough for that to matter. Now that performance has improved,
write timeouts are more likely to occur. We should handle these by backing
off and retrying the operation.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9274) Changing memtable_flush_writes per recommendations in cassandra.yaml causes memtable_cleanup_threshold to be too small

2015-04-30 Thread Donald Smith (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Donald Smith updated CASSANDRA-9274:

Description: 
It says in cassandra.yaml:
{noformat}
# If your data directories are backed by SSD, you should increase this
# to the number of cores.
#memtable_flush_writers: 8
{noformat}
so we raised it to 24.

Much later we noticed a warning in the logs:
{noformat}
WARN  [main] 2015-04-22 15:32:58,619 DatabaseDescriptor.java:539 - 
memtable_cleanup_threshold is set very low, which may cause performance 
degradation
{noformat}
Looking at cassandra.yaml again I see:
{noformat}
# memtable_cleanup_threshold defaults to 1 / (memtable_flush_writers + 1)
# memtable_cleanup_threshold: 0.11
#memtable_cleanup_threshold: 0.11
{noformat}
So, I uncommented that last line (figuring that 0.11 is a reasonable value).

Cassandra.yaml should give better guidance or the code should *prevent* the 
value from going outside a reasonable range.


  was:
It says in cassandra.yaml:
{noformat}
# If your data directories are backed by SSD, you should increase this
# to the number of cores.
#memtable_flush_writers: 8
{noformat}
so we raised it to 24.

Much later we noticed a warning in the logs:
{noformat}
WARN  [main] 2015-04-22 15:32:58,619 DatabaseDescriptor.java:539 - 
memtable_cleanup_threshold is set very low, which may cause performance 
degradation
{noformat}
Looking at cassandra.yaml again I see:
{noformat}
# memtable_cleanup_threshold defaults to 1 / (memtable_flush_writers + 1)
# memtable_cleanup_threshold: 0.11
#memtable_cleanup_threshold: 0.11
{noformat}
So, I uncommented that last line (figuring that 0.11 is a reasonable value).

Cassandra.yaml should give better guidance or the code should *prevent* the 
value from going outside a reasonable range.
{noformat}


 Changing memtable_flush_writes per recommendations in cassandra.yaml causes  
 memtable_cleanup_threshold to be too small
 ---

 Key: CASSANDRA-9274
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9274
 Project: Cassandra
  Issue Type: Improvement
Reporter: Donald Smith
Priority: Minor

 It says in cassandra.yaml:
 {noformat}
 # If your data directories are backed by SSD, you should increase this
 # to the number of cores.
 #memtable_flush_writers: 8
 {noformat}
 so we raised it to 24.
 Much later we noticed a warning in the logs:
 {noformat}
 WARN  [main] 2015-04-22 15:32:58,619 DatabaseDescriptor.java:539 - 
 memtable_cleanup_threshold is set very low, which may cause performance 
 degradation
 {noformat}
 Looking at cassandra.yaml again I see:
 {noformat}
 # memtable_cleanup_threshold defaults to 1 / (memtable_flush_writers + 1)
 # memtable_cleanup_threshold: 0.11
 #memtable_cleanup_threshold: 0.11
 {noformat}
 So, I uncommented that last line (figuring that 0.11 is a reasonable value).
 Cassandra.yaml should give better guidance or the code should *prevent* the 
 value from going outside a reasonable range.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9275) ReadMessageTest.testNoCommitLog fails with test-compression


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9275:
--
Fix Version/s: 3.x

 ReadMessageTest.testNoCommitLog fails with test-compression
 ---

 Key: CASSANDRA-9275
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9275
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Branimir Lambov
 Fix For: 3.x


 It's reading the raw bytes of the commit log expecting to find an 
 uncompressed log and instead finding a compressed log and now the test is not 
 working if the log is compressed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6477) Global indexes

2015-04-30 Thread Matthias Broecheler (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522171#comment-14522171
]

Matthias Broecheler commented on CASSANDRA-6477:

bq. Sorry for a short reply. Just wanted to note that I see this being the
other way around. Full-on materialized views are the infrastructure for 'global
indexes', not the other way around.

Do you mean that from an implementation perspective? Conceptually, I would
argue it the other way around. When you build support for full-on materialized
views, you still have to deal with the decision: is the primary key high or low
selectivity, i.e. do I maintain this view locally or globally. And then,
depending on that decision, you would utilize the 2i infrastructure (or
something like it) or some other infrastructure for global maintenance of
derived records.

Plus, with full-on materialized views you are mixing in other challenges, like
dealing with multiple columns (e.g. CASSANDRA-5402).
So it seems, conceptually at least, that the initial atomic problem seems to be
how to consistently maintain derived row records on remote nodes. That would be
enough to build global indexes on a single column. From there, this could be
extended to full-on materialized view maintenance.

Global indexes
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9261) Prepare and Snapshot for repairs should use higher timeouts for expiring map

2015-04-30 Thread sankalp kohli (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522023#comment-14522023
 ] 

sankalp kohli commented on CASSANDRA-9261:
--

[~yukim] Do you need a new patch for 2.1. If yes it might take time.  

 Prepare and Snapshot for repairs should use higher timeouts for expiring map
 

 Key: CASSANDRA-9261
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9261
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli
Assignee: sankalp kohli
Priority: Minor
 Attachments: trunk_9261.txt


 We wait for 1 hour after sending the prepare message but expiring map will 
 remove it after RPC timeout. 
 In snapshot during repair, we only wait for RPC time. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9273) Avoid calculating maxPurgableTimestamp for partitions with non-expired TTL

Marcus Eriksson created CASSANDRA-9273:
--

 Summary: Avoid calculating maxPurgableTimestamp for partitions 
with non-expired TTL
 Key: CASSANDRA-9273
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9273
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
 Fix For: 2.1.x


Seems we still calculate maxPurgableTimestamp for partitions where we have TTL 
that is known to not be expired, we should try to avoid that



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9272) Atomic Batch Statements Fail to Execute on Some Coordinators


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9272:
---
 Reviewer:   (was: Philip Thompson)
Reproduced In: 2.0.14, 2.0.6  (was: 2.0.6, 2.0.14)
 Assignee: Carl Yeksigian

I can reproduce this easily with a simple 2 dc, 3 node ccm cluster, using the 
exact keyspace, table, and commands given by the reporter.

 Atomic Batch Statements Fail to Execute on Some Coordinators
 

 Key: CASSANDRA-9272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9272
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Operating System: Red Hat Enterprise Linux Server 
 release 6.5 (Santiago) |
 Cassandra and CQL: [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift 
 protocol 19.39.0] |
 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_79 (Oracle Java) |
 Using PropertyFileSnitch to define cluster topology.
Reporter: Jared Gray
Assignee: Carl Yeksigian
 Fix For: 2.0.x


 Install and configure a three-node Cassandra cluster spread across two data 
 centers (DCs). Place two Cassandra instances in DC1 and one Cassandra 
 instance in DC2. (We have designated one node as a seed in each data center.)
 Gracefully shut down any one Cassandra instance in DC1. This can be done by 
 running 'kill process_id' in Linux.
 Open a CQL Shell connection to the remaining node in DC1.
 Create the following keyspace: CREATE KEYSPACE batch_test WITH replication = 
 {'class': 'NetworkTopologyStrategy', 'DC1': '2', 'DC2':'1'};
 Create the following table: CREATE TABLE batch_test.batch_table ( 
 partition_key TEXT PRIMARY KEY );
 Execute the following atomic batch query: BEGIN BATCH INSERT INTO 
 batch_test.batch_table (partition_key) VALUES ('a'); INSERT INTO 
 batch_test.batch_table (partition_key) VALUES ('b'); APPLY BATCH;
 Observe an availability-related error, despite a sufficient number of nodes 
 available to service the query. The trace below shows sample results from our 
 cluster:
 DC1
 
 cqlsh CONSISTENCY
 Current consistency level is ONE.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 Unable to complete request: one or more nodes were unavailable.
 cqlsh CONSISTENCY QUORUM
 Consistency level set to QUORUM.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 Unable to complete request: one or more nodes were unavailable.
 cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('a'); INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('b'); APPLY BATCH;
 cqlsh
 Note: an UNLOGGED BATCH executes successfully. A regular BATCH also executes 
 successfully if it only contains one INSERT query.
 We now show a similar trace using a CQL Shell connected to the Cassandra 
 instance in DC2:
 DC2
 ---
 cqlsh CONSISTENCY
 Current consistency level is ONE.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 cqlsh CONSISTENCY QUORUM
 Consistency level set to QUORUM.
 cqlsh BEGIN BATCH INSERT INTO batch_test.batch_table (partition_key) VALUES 
 ('a'); INSERT INTO batch_test.batch_table (partition_key) VALUES ('b'); APPLY 
 BATCH;
 cqlsh BEGIN UNLOGGED BATCH INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('a'); INSERT INTO batch_test.batch_table 
 (partition_key) VALUES ('b'); APPLY BATCH;
 cqlsh
 All queries execute successfully using the DC2 Cassandra instance as a 
 coordinator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9276) StreamingTransferTest fails under test-compression due to bad assertion


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9276:
--
Fix Version/s: 3.x

 StreamingTransferTest fails under test-compression due to bad assertion
 ---

 Key: CASSANDRA-9276
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9276
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.x


 https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
 {noformat}
 assert in.getBytesRead()  totalSize;
 {noformat]
 My guess is that total size is the compressed size, not the uncompressed 
 size. Remove the assertion and the test passes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9276) StreamingTransferTest fails under test-compression due to bad assertion


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9276:
--
Description: 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
{noformat}
assert in.getBytesRead()  totalSize;
{noformat}
My guess is that total size is the compressed size, not the uncompressed size. 
Remove the assertion and the test passes.

  was:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
{noformat}
assert in.getBytesRead()  totalSize;
{noformat]
My guess is that total size is the compressed size, not the uncompressed size. 
Remove the assertion and the test passes.


 StreamingTransferTest fails under test-compression due to bad assertion
 ---

 Key: CASSANDRA-9276
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9276
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.x


 https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
 {noformat}
 assert in.getBytesRead()  totalSize;
 {noformat}
 My guess is that total size is the compressed size, not the uncompressed 
 size. Remove the assertion and the test passes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9271) IndexSummaryManagerTest.testCompactionRace times out periodically


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9271:
--
Fix Version/s: 3.x

 IndexSummaryManagerTest.testCompactionRace times out periodically
 -

 Key: CASSANDRA-9271
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9271
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
 Fix For: 3.x


 The issue is that the amount of time the test takes is highly variable to it 
 being biased towards creating a condition where the test has to retry the 
 compaction it is attempting.
 Solution is to decrease the bias by having 
 https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L2522
  check every millisecond instead of every 100 milliseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)

2015-04-30 Thread graham sanderson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522753#comment-14522753
 ] 

graham sanderson commented on CASSANDRA-8325:
-

My suggestion would be to see if you can perf test this yourself on Linux. 
Benedict is right that it might pollute the instruction cache, but then it 
might not (it really depends on the mix of calls).. I think the point is that 
someone who needs FreeBSD is probably going to have to do the leg work here; or 
just run with my patch which may be sub-optimal.

 Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
 -

 Key: CASSANDRA-8325
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325
 Project: Cassandra
  Issue Type: Bug
 Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit 
 Server VM
Reporter: Leonid Shalupov
 Fix For: 2.1.x

 Attachments: hs_err_pid1856.log, system.log, unsafeCopy1.txt, 
 untested_8325.patch


 See attached error file after JVM crash
 {quote}
 FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu 
 Jan 16 22:34:59 UTC 2014 
 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
 {quote}
 {quote}
  % java -version
 openjdk version 1.7.0_71
 OpenJDK Runtime Environment (build 1.7.0_71-b14)
 OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode)
 {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9279) Gossip locks up on startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebastian Estevez updated CASSANDRA-9279:
-
Description: 
Cluster running 2.0.14.352 on EC2 - c3.4xl's

2 nodes out of 8 exhibited the following behavior

When starting up the node we noticed it was gray in OpsCenter. Other monitoring 
tool showed it as up. 

Turned out gossip tasks were piling up and we could see the following in the 
system.log:

{code}

GossipStage:1 daemon prio=10 tid=0x7ffa23471800 nid=0xa13fa waiting on 
condition [0x7ff9cbe26000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x0005d3f50960 (a 
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
at 
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:731)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:351)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:336)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
at 
org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:709)
at 
org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
at 
org.apache.cassandra.db.SystemKeyspace.updatePeerInfo(SystemKeyspace.java:379)
- locked 0x0005d3f41ed8 (a java.lang.Class for 
org.apache.cassandra.db.SystemKeyspace)
at 
org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:1414)
at 
org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1524)
at 
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1350)
at 
org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1083)
at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:1065)
at 
org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1023)
at 
org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:58)
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

   Locked ownable synchronizers:
- 0x000609517438 (a 
java.util.concurrent.ThreadPoolExecutor$Worker)
...
{code}

and tpstats shows blocked tasks--gossip and mutations:

{code}
GossipStage   1  3904  29384 0  
   0
{code}

the CPU's are inactive (See attachment)

and dstat output:

{code}
You did not select any stats, using -cdngy by default.
total-cpu-usage -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  2   0  97   0   0   0|1324k 1381k|   0 0 |   0 0 |6252  5548
  0   0 100   0   0   0|   064k|  42k 1017k|   0 0 |3075  2537
  0   0  99   0   0   0|   0  8192B|  39k  794k|   0 0 |6999  7039
  0   0 100   0   0   0|   0 0 |  39k  759k|   0 0 |3067  2726
  0   0  99   0   0   0|   0   184k|  48k 1086k|   0 0 |4829  4178
  0   0  99   0   0   0|   0  8192B|  34k  802k|   0 0 |1671  1240
  0   0 100   0   0   0|   0  8192B|  48k 1067k|   0 0 |1878  1193
{code}

I managed to grab a thread dump:
https://gist.githubusercontent.com/anonymous/3b7b4698c32032603493/raw/read.md

and dmesg:
https://gist.githubusercontent.com/anonymous/5982b15337c9afbd5d49/raw/f3c2e4411b9d59e90f4615d93c7c1ad25922e170/read.md

Restarting the node solved the issue (it came up normally), we don't know what 
is causing it but apparently (per the thread dump) gossip threads are blocked 
writing the system keyspace and the writes waiting on the commitlog.


Gossip:
{code}
GossipTasks:1 daemon prio=10 tid=0x7ffa2368f800 nid=0xa140e runnable 
[0x7ffc16fb2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x0005d4378b88 (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
{code}

Mutation:
{code}
MutationStage:32 daemon prio=10

[jira] [Updated] (CASSANDRA-9261) Prepare and Snapshot for repairs should use higher timeouts for expiring map


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-9261:
--
Attachment: 0001-make-prepare-snapshot-timeout-to-1-hour.patch

I attached patch for 2.1, with snapshot timeout changed to 1 hour as I proposed 
above.
[~kohlisankalp] If you are ok with time out value, I'm ready to commit.

 Prepare and Snapshot for repairs should use higher timeouts for expiring map
 

 Key: CASSANDRA-9261
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9261
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli
Assignee: sankalp kohli
Priority: Minor
 Attachments: 0001-make-prepare-snapshot-timeout-to-1-hour.patch, 
 trunk_9261.txt


 We wait for 1 hour after sending the prepare message but expiring map will 
 remove it after RPC timeout. 
 In snapshot during repair, we only wait for RPC time. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9196) Do not rebuild indexes if no columns are actually indexed

2015-04-30 Thread Jeremiah Jordan (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522693#comment-14522693
 ] 

Jeremiah Jordan commented on CASSANDRA-9196:


This looks like it may be broken for indexes on DynamicCompositeType columns.  
Still looking into it, but thought I would post a heads up.

 Do not rebuild indexes if no columns are actually indexed
 -

 Key: CASSANDRA-9196
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9196
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Sergio Bossa
Assignee: Sergio Bossa
 Fix For: 2.0.15, 2.1.5

 Attachments: 2.0-CASSANDRA-9196.txt, 2.1-CASSANDRA-9196.txt


 When rebuilding secondary indexes, the index task is executed regardless if 
 the actual {{SecondaryIndex#indexes(ByteBuffer )}} implementation of any 
 index returns true for any column, meaning that the expensive task of going 
 through all sstables and related rows will be executed even if in the end no 
 column/row will be actually indexed.
 This is a huge performance hit when i.e. bootstrapping with large datasets on 
 tables having custom secondary index implementations whose {{indexes()}} 
 implementation might return false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9261) Prepare and Snapshot for repairs should use higher timeouts for expiring map

2015-04-30 Thread sankalp kohli (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522613#comment-14522613
 ] 

sankalp kohli commented on CASSANDRA-9261:
--

+1

 Prepare and Snapshot for repairs should use higher timeouts for expiring map
 

 Key: CASSANDRA-9261
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9261
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli
Assignee: sankalp kohli
Priority: Minor
 Attachments: 0001-make-prepare-snapshot-timeout-to-1-hour.patch, 
 trunk_9261.txt


 We wait for 1 hour after sending the prepare message but expiring map will 
 remove it after RPC timeout. 
 In snapshot during repair, we only wait for RPC time. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9279) Gossip locks up on startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebastian Estevez updated CASSANDRA-9279:
-
Description: 
Cluster running 2.0.14.352 on EC2 - c3.4xl's

2 nodes out of 8 exhibited the following behavior

When starting up the node we noticed it was gray in OpsCenter. Other monitoring 
tool showed it as up. 

Turned out gossip tasks were piling up and we could see the following in the 
system.log:

{code}
 WARN [GossipTasks:1] 2015-04-30 20:22:29,512 Gossiper.java (line 671) Gossip 
stage has 4270 pending tasks; skipping status check (no nodes will be marked 
down)
 WARN [GossipTasks:1] 2015-04-30 20:22:30,612 Gossiper.java (line 671) Gossip 
stage has 4272 pending tasks; skipping status check (no nodes will be marked 
down)
 WARN [GossipTasks:1] 2015-04-30 20:22:31,713 Gossiper.java (line 671) Gossip 
stage has 4273 pending tasks; skipping status check (no nodes will be marked 
down)
...
{code}

and tpstats shows blocked tasks--gossip and mutations:

{code}
GossipStage   1  3904  29384 0  
   0
{code}

the CPU's are inactive (See attachment)

and dstat output:

{code}
You did not select any stats, using -cdngy by default.
total-cpu-usage -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  2   0  97   0   0   0|1324k 1381k|   0 0 |   0 0 |6252  5548
  0   0 100   0   0   0|   064k|  42k 1017k|   0 0 |3075  2537
  0   0  99   0   0   0|   0  8192B|  39k  794k|   0 0 |6999  7039
  0   0 100   0   0   0|   0 0 |  39k  759k|   0 0 |3067  2726
  0   0  99   0   0   0|   0   184k|  48k 1086k|   0 0 |4829  4178
  0   0  99   0   0   0|   0  8192B|  34k  802k|   0 0 |1671  1240
  0   0 100   0   0   0|   0  8192B|  48k 1067k|   0 0 |1878  1193
{code}

I managed to grab a thread dump:
https://gist.githubusercontent.com/anonymous/3b7b4698c32032603493/raw/read.md

and dmesg:
https://gist.githubusercontent.com/anonymous/5982b15337c9afbd5d49/raw/f3c2e4411b9d59e90f4615d93c7c1ad25922e170/read.md

Restarting the node solved the issue (it came up normally), we don't know what 
is causing it but apparently (per the thread dump) gossip threads are blocked 
writing the system keyspace and the writes waiting on the commitlog.


Gossip:
{code}
GossipStage:1 daemon prio=10 tid=0x7ffa23471800 nid=0xa13fa waiting on 
condition [0x7ff9cbe26000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x0005d3f50960 (a 
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
at 
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:731)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:351)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:336)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
at 
org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:709)
at 
org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
at 
org.apache.cassandra.db.SystemKeyspace.updatePeerInfo(SystemKeyspace.java:379)
- locked 0x0005d3f41ed8 (a java.lang.Class for 
org.apache.cassandra.db.SystemKeyspace)
at 
org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:1414)
at 
org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1524)
at 
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1350)
at 
org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1083)
at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:1065)
at 
org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1023)
at 
org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:58)
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

   Locked ownable synchronizers:
- 0x000609517438 (a

[jira] [Commented] (CASSANDRA-9267) Bootstrapping node, nodetool netstats shows all files 100% received immediately

2015-04-30 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522398#comment-14522398
 ] 

Brandon Williams commented on CASSANDRA-9267:
-

First I'd see if this reproduces on 2.0.14 so you aren't wasting your time.  
Then I'd look at the appropriate calls in the nodetool source itself (all the 
Node* files in o.a.c.tools) and then StreamManager.java since it's just passing 
the raw info to nodetool.

 Bootstrapping node, nodetool netstats shows all files 100% received 
 immediately
 ---

 Key: CASSANDRA-9267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9267
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Centos
Reporter: Derek Troy-West
Priority: Minor
  Labels: bootstrap, nodetool, streaming
 Fix For: 2.0.x


 Using Cassandra v.2.0.10.
 Following the progress of a new node bootstrapping with 'nodetool netstats' I 
 notice that each file is shown as 100% received immediately.
 i.e.
 {code}
 $ nodetool netstats -h 172.16.11.17
 Mode: JOINING
 Bootstrap 9ec3ab30-eef1-11e4-a7f2-51821d8b858c
 /172.16.11.16
 /172.16.11.12
 Receiving 103 files, 110808118132 bytes total
 /data/cassandra/01/mail/archive/mail-archive-tmp-jb-49-Data.db 
 5388827962/5388827962 bytes(100%) received from /172.16.11.12
 
 /data/cassandra/02/mail/msg_summary_subject_idx/mail-msg_summary_subject_idx-tmp-jb-11-Data.db
  88344414/88344414 bytes(100%) received from /172.16.11.12
 /data/cassandra/02/OpsCenter/pdps/OpsCenter-pdps-tmp-jb-8-Data.db 
 323/323 bytes(100%) received from /172.16.11.12
 /data/cassandra/02/mail/archive/mail-archive-tmp-jb-63-Data.db 
 10470429245/10470429245 bytes(100%) received from /172.16.11.12
 /data/cassandra/03/mail/mail_log/mail-mail_log-tmp-jb-17-Data.db 
 2846783513/2846783513 bytes(100%) received from /172.16.11.12
 {code}
 The node is bootstrapping 200GB in total, and quite quickly/painlessly as far 
 as I can tell, but each file is shown as 100% transferred already.
 Since this is fairly minor I'm happy to hunt it down and patch it myself if 
 desirable. Any pointers where to start appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9207) COPY FROM command does not restore all records


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522449#comment-14522449
 ] 

Tyler Hobbs commented on CASSANDRA-9207:


Yes, the precision issue affects doubles as well.  To increase the precision, 
in your cqlshrc file, add the following:

{noformat}
[ui]
float_precision = 16
{noformat}

(or use whatever precision you want).  On a related note, CASSANDRA-9224 should 
improve the default precision settings.



 COPY FROM command does not restore all records
 --

 Key: CASSANDRA-9207
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9207
 Project: Cassandra
  Issue Type: Bug
  Components: Core, Tools
 Environment: centOs, cassandra 2.1.1
Reporter: Gaurav
Assignee: Tyler Hobbs
  Labels: COPY, cqlsh
 Fix For: 2.1.x

 Attachments: cron_task.csv


 Steps to reproduce:
 1. run COPY TO command on keyspace A. (on my environment problem occurred on 
 only 1 table out of 19, only thing that io noticed is this table has 1500 
 records and other tables has 1000 records)
 2. pull the csv file and place it on machine (say machine B)where  other 
 database resides. (In my case this was another instance of amazon machine)
 3. Now, run the COPY TO command on machine B. (both keyspaces, one on machine 
 A and one on machine B has same schema.)
 Observation:
 1. when COPY TO command is run for table having records  1500. Command gave 
 following output:
 Processing 1000 records
 1573 records copied.
 but when i tried to verify it by running below mentioned commands, i received 
 only 273 records.
 1. SELECT COUNT(*) FROM table-name;
 2. SELECT * FROM table-name;
 Note: please let me know if other information needs to be shared with you.
 Also, is there any other way to take the back-up of keyspace and restoring it 
 on other machine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9277) SSTableRewriterTest.testFileRemoval fails with test-compression

Ariel Weisberg created CASSANDRA-9277:
-

 Summary: SSTableRewriterTest.testFileRemoval fails with 
test-compression
 Key: CASSANDRA-9277
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9277
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.x


openEarly returns null with compression because not enough data has been 
written to trigger a flush. Solution is to write more data in the test case so 
it flushes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9279) Gossip locks up on startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebastian Estevez updated CASSANDRA-9279:
-
Description: 
Cluster running 2.0.14.352 on EC2 - c3.4xl's

2 nodes out of 8 exhibited the following behavior

When starting up the node we noticed it was gray in OpsCenter. Other monitoring 
tool showed it as up. 

Turned out gossip tasks were piling up and we could see the following in the 
system.log:

{code}
 WARN [GossipTasks:1] 2015-04-30 20:22:29,512 Gossiper.java (line 671) Gossip 
stage has 4270 pending tasks; skipping status check (no nodes will be marked 
down)
 WARN [GossipTasks:1] 2015-04-30 20:22:30,612 Gossiper.java (line 671) Gossip 
stage has 4272 pending tasks; skipping status check (no nodes will be marked 
down)
 WARN [GossipTasks:1] 2015-04-30 20:22:31,713 Gossiper.java (line 671) Gossip 
stage has 4273 pending tasks; skipping status check (no nodes will be marked 
down)
...
{code}

and tpstats shows blocked tasks--gossip and mutations:

{code}
GossipStage   1  3904  29384 0  
   0
{code}

the CPU's are inactive (See attachment)

and dstat output:

{code}
You did not select any stats, using -cdngy by default.
total-cpu-usage -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  2   0  97   0   0   0|1324k 1381k|   0 0 |   0 0 |6252  5548
  0   0 100   0   0   0|   064k|  42k 1017k|   0 0 |3075  2537
  0   0  99   0   0   0|   0  8192B|  39k  794k|   0 0 |6999  7039
  0   0 100   0   0   0|   0 0 |  39k  759k|   0 0 |3067  2726
  0   0  99   0   0   0|   0   184k|  48k 1086k|   0 0 |4829  4178
  0   0  99   0   0   0|   0  8192B|  34k  802k|   0 0 |1671  1240
  0   0 100   0   0   0|   0  8192B|  48k 1067k|   0 0 |1878  1193
{code}

I managed to grab a thread dump:
https://gist.githubusercontent.com/anonymous/3b7b4698c32032603493/raw/read.md

and dmesg:
https://gist.githubusercontent.com/anonymous/5982b15337c9afbd5d49/raw/f3c2e4411b9d59e90f4615d93c7c1ad25922e170/read.md

Restarting the node solved the issue (it came up normally), we don't know what 
is causing it but apparently (per the thread dump) gossip threads are blocked 
writing the system keyspace and the writes waiting on the commitlog.


Gossip:
{code}
GossipTasks:1 daemon prio=10 tid=0x7ffa2368f800 nid=0xa140e runnable 
[0x7ffc16fb2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x0005d4378b88 (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
{code}

Mutation:
{code}
MutationStage:32 daemon prio=10 tid=0x7ffa2339c800 nid=0xa1399 waiting on 
condition [0x7ff9cd6c8000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x0005d486a888 (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at 
java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:349)
{code}

  was:
Cluster running 2.0.14.352 on EC2 - c3.4xl's

2 nodes out of 8 exhibited the following behavior

When starting up the node we noticed it was gray in OpsCenter. Other monitoring 
tool showed it as up. 

Turned out gossip tasks were piling up and we could see the following in the 
system.log:

{code}
 WARN [GossipTasks:1] 2015-04-30 20:22:29,512 Gossiper.java (line 671) Gossip 
stage has 4270 pending tasks; skipping status check (no nodes will be marked 
down)
 WARN [GossipTasks:1] 2015-04-30 20:22:30,612 Gossiper.java (line 671) Gossip 
stage has 4272 pending tasks; skipping status check (no nodes will be marked 
down)
 WARN [GossipTasks:1] 2015-04-30 20:22:31,713 Gossiper.java (line 671) Gossip 
stage has 4273 pending tasks; skipping status check (no nodes will be marked 
down)
...
{code}

and tpstats shows blocked tasks--gossip and mutations:

{code}
GossipStage   1  3904  29384 0  
   0
{code}

the CPU's are inactive (See attachment)

and dstat output:

{code}
You did not select any stats, using -cdngy by default.
total-cpu-usage -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  2   0  97   0   0   0|1324k 1381k|   0 0 |   0 0 |6252  5548
  0   0 100   0   0   0|   064k|  42k 1017k|   0 0 |3075  2537
  0   0  99   0   0   0|   0  8192B|  39k  794k|   0 0 |6999  7039
  0   0 100   0   0   0|   0 0 |  39k

[jira] [Updated] (CASSANDRA-9279) Gossip (and mutations) lock up on startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebastian Estevez updated CASSANDRA-9279:
-
Summary: Gossip (and mutations) lock up on startup  (was: Gossip locks up 
on startup)

 Gossip (and mutations) lock up on startup
 -

 Key: CASSANDRA-9279
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9279
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
 Attachments: Screen Shot 2015-04-30 at 4.41.57 PM.png


 Cluster running 2.0.14.352 on EC2 - c3.4xl's
 2 nodes out of 8 exhibited the following behavior
 When starting up the node we noticed it was gray in OpsCenter. Other 
 monitoring tool showed it as up. 
 Turned out gossip tasks were piling up and we could see the following in the 
 system.log:
 {code}
  WARN [GossipTasks:1] 2015-04-30 20:22:29,512 Gossiper.java (line 671) Gossip 
 stage has 4270 pending tasks; skipping status check (no nodes will be marked 
 down)
  WARN [GossipTasks:1] 2015-04-30 20:22:30,612 Gossiper.java (line 671) Gossip 
 stage has 4272 pending tasks; skipping status check (no nodes will be marked 
 down)
  WARN [GossipTasks:1] 2015-04-30 20:22:31,713 Gossiper.java (line 671) Gossip 
 stage has 4273 pending tasks; skipping status check (no nodes will be marked 
 down)
 ...
 {code}
 and tpstats shows blocked tasks--gossip and mutations:
 {code}
 GossipStage   1  3904  29384 0
  0
 {code}
 the CPU's are inactive (See attachment)
 and dstat output:
 {code}
 You did not select any stats, using -cdngy by default.
 total-cpu-usage -dsk/total- -net/total- ---paging-- ---system--
 usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
   2   0  97   0   0   0|1324k 1381k|   0 0 |   0 0 |6252  5548
   0   0 100   0   0   0|   064k|  42k 1017k|   0 0 |3075  2537
   0   0  99   0   0   0|   0  8192B|  39k  794k|   0 0 |6999  7039
   0   0 100   0   0   0|   0 0 |  39k  759k|   0 0 |3067  2726
   0   0  99   0   0   0|   0   184k|  48k 1086k|   0 0 |4829  4178
   0   0  99   0   0   0|   0  8192B|  34k  802k|   0 0 |1671  1240
   0   0 100   0   0   0|   0  8192B|  48k 1067k|   0 0 |1878  1193
 {code}
 I managed to grab a thread dump:
 https://gist.githubusercontent.com/anonymous/3b7b4698c32032603493/raw/read.md
 and dmesg:
 https://gist.githubusercontent.com/anonymous/5982b15337c9afbd5d49/raw/f3c2e4411b9d59e90f4615d93c7c1ad25922e170/read.md
 Restarting the node solved the issue (it came up normally), we don't know 
 what is causing it but apparently (per the thread dump) gossip threads are 
 blocked writing the system keyspace and the writes waiting on the commitlog.
 Gossip:
 {code}
 GossipStage:1 daemon prio=10 tid=0x7ffa23471800 nid=0xa13fa waiting on 
 condition [0x7ff9cbe26000]
java.lang.Thread.State: WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  0x0005d3f50960 (a 
 java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
   at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964)
   at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
   at 
 java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:731)
   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:351)
   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:336)
   at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
   at 
 org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:709)
   at 
 org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
   at 
 org.apache.cassandra.db.SystemKeyspace.updatePeerInfo(SystemKeyspace.java:379)
   - locked 0x0005d3f41ed8 (a java.lang.Class for 
 org.apache.cassandra.db.SystemKeyspace)
   at 
 org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:1414)
   at 
 org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1524)
   at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:1350)
   at 
 org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1083)
   at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:1065)
   at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1023)
   at

[jira] [Resolved] (CASSANDRA-9207) COPY FROM command does not restore all records


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs resolved CASSANDRA-9207.

Resolution: Not A Problem

 COPY FROM command does not restore all records
 --

 Key: CASSANDRA-9207
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9207
 Project: Cassandra
  Issue Type: Bug
  Components: Core, Tools
 Environment: centOs, cassandra 2.1.1
Reporter: Gaurav
Assignee: Tyler Hobbs
  Labels: COPY, cqlsh
 Fix For: 2.1.x

 Attachments: cron_task.csv


 Steps to reproduce:
 1. run COPY TO command on keyspace A. (on my environment problem occurred on 
 only 1 table out of 19, only thing that io noticed is this table has 1500 
 records and other tables has 1000 records)
 2. pull the csv file and place it on machine (say machine B)where  other 
 database resides. (In my case this was another instance of amazon machine)
 3. Now, run the COPY TO command on machine B. (both keyspaces, one on machine 
 A and one on machine B has same schema.)
 Observation:
 1. when COPY TO command is run for table having records  1500. Command gave 
 following output:
 Processing 1000 records
 1573 records copied.
 but when i tried to verify it by running below mentioned commands, i received 
 only 273 records.
 1. SELECT COUNT(*) FROM table-name;
 2. SELECT * FROM table-name;
 Note: please let me know if other information needs to be shared with you.
 Also, is there any other way to take the back-up of keyspace and restoring it 
 on other machine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9248) Tests for cqlsh parser