[jira] [Commented] (CASSANDRA-9628) "Unknown keyspace system_traces" exception when using nodetool on a new cluster

[David Wood (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057240#comment-15057240
 ] 

David Wood commented on CASSANDRA-9628:
---

Hello,

I'm using Cassandra v2.1.8.  I can re-create this problem if I include the 
following settings in cassandra.yaml:

{code}
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
{code}

If I comment them out and re-start Cassandra then the problem goes away.  
Strangely, the above settings are supposedly the defaults:

https://docs.datastax.com/en/cassandra/2.1/cassandra/configuration/configCassandra_yaml_r.html

David


> "Unknown keyspace system_traces" exception when using nodetool on a new 
> cluster
> ---
>
> Key: CASSANDRA-9628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9628
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: tzach
>Assignee: Carl Yeksigian
>Priority: Minor
>
> When creating a new cluster from scratch,  nodetool status fails on 
> system_traces as follow
> {code}
> $ nodetool status
> error: Unknown keyspace system_traces
> -- StackTrace --
> java.lang.AssertionError: Unknown keyspace system_traces
>   at org.apache.cassandra.db.Keyspace.(Keyspace.java:270)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:119)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:96)
> ...
> {code}
> the problem disappear when creating an empty keyspace
> {code}
> cqlsh> create keyspace temp WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 2 };
> {code}
> My guess is system_traces initialization complete only after any data 
> insertion.
> Before it does, any attempt to read  from it either from nodetool, cqlsh or 
> streaming to a new node will fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057107#comment-15057107
 ] 

Yuki Morishita commented on CASSANDRA-10859:


CASSANDRA-10679 added "-ea" to execute commands.
Adding "-ea" to pre CASSANDRA-10679 nodetool resulted in the same 
AssertionError as we are observing right now.

I feel removing "-ea" as before is not the right direction to take here, so I 
will continue to investigate the correct fix to the problem.

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x, 3.x
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/263763ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/263763ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/263763ab

Branch: refs/heads/cassandra-2.2
Commit: 263763ab9bfd9be902951d06f5d44f3a9d3f6f4f
Parents: 6ff7c9c 9b30d65
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:27:21 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:28:05 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/263763ab/CHANGES.txt
--
diff --cc CHANGES.txt
index d44a47b,091ac52..592ba0a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,12 +1,35 @@@
 -2.1.13
 +2.2.5
 + * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 + * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 + * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 + * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 + * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
 +Merged from 2.1:
+  * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
 - * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
   * Fix Stress profile parsing on Windows (CASSANDRA-10808)
  
 -
 -2.1.12
 +2.2.4
 + * Show CQL help in cqlsh in web browser (CASSANDRA-7225)
 + * Serialize on disk the proper SSTable compression ratio (CASSANDRA-10775)
 + * Reject index queries while the index is building (CASSANDRA-8505)
 + * CQL.textile syntax incorrectly includes optional keyspace for aggregate 
SFUNC and FINALFUNC (CASSANDRA-10747)
 + * Fix JSON update with prepared statements (CASSANDRA-10631)
 + * Don't do anticompaction after subrange repair (CASSANDRA-10422)
 + * Fix SimpleDateType type compatibility (CASSANDRA-10027)
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 + * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
 +Merged from 2.1:
   * Fix incremental repair hang when replica is down (CASSANDRA-10288)
   * Avoid writing range tombstones after END_OF_ROW marker (CASSANDRA-10791)
   * Optimize the way we check if a token is repaired in anticompaction 
(CASSANDRA-10768)

[1/2] cassandra git commit: Make Stress compiles within eclipse

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 6ff7c9ce9 -> 263763ab9


Make Stress compiles within eclipse

patch by Benjamin Lerer; reviewed by Jake Luciani for CASSANDRA-10807


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b30d657
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b30d657
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b30d657

Branch: refs/heads/cassandra-2.2
Commit: 9b30d6572fdb988796788ad4c7b8daabdef4e961
Parents: dff2214
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:17:50 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:22:30 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 96f2f4f..091ac52 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.13
+ * Make Stress compiles within eclipse (CASSANDRA-10807)
  * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
  * Allow cancellation of index summary redistribution (CASSANDRA-8805)
  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
index 00f866a..6b39d08 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
@@ -31,13 +31,13 @@ public abstract class Generator
 
 public final String name;
 public final AbstractType type;
-public final Class clazz;
+public final Class clazz;
 final long salt;
 final Distribution identityDistribution;
 final Distribution sizeDistribution;
 public final Distribution clusteringDistribution;
 
-public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
+public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
 {
 this.type = type;
 this.name = name;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
index bfa58ea..d82e01f 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
+++ b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
@@ -26,16 +26,17 @@ import java.util.List;
 
 import org.apache.cassandra.db.marshal.ListType;
 
-public class Lists extends Generator
+public class Lists extends Generator
 {
-final Generator valueType;
-final Object[] buffer;
+final Generator valueType;
+final T[] buffer;
 
-public Lists(String name, Generator valueType, GeneratorConfig config)
+@SuppressWarnings("unchecked")
+public Lists(String name, Generator valueType, GeneratorConfig config)
 {
 super(ListType.getInstance(valueType.type, true), config, name, 
List.class);
 this.valueType = valueType;
-buffer = new Object[(int) sizeDistribution.maxValue()];
+buffer = (T[]) new Object[(int) sizeDistribution.maxValue()];
 }
 
 public void setSeed(long seed)
@@ -45,7 +46,7 @@ public class Lists extends Generator
 }
 
 @Override
-public List generate()
+public List generate()
 {
 int size = (int) sizeDistribution.next();
 for (int i = 0 ; i < size ; i++)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
index 5c17b4e..b74d6b2 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
+++ 

cassandra git commit: Fix missing row data after range tombstone in legacy data

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 94e8d62cf -> 71bac92cf


Fix missing row data after range tombstone in legacy data

patch by blambov; reviewed by slebresne for CASSANDRA-10822


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/71bac92c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/71bac92c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/71bac92c

Branch: refs/heads/cassandra-3.0
Commit: 71bac92cf1df08de194a8283f382a7950b3a78ed
Parents: 94e8d62
Author: Branimir Lambov 
Authored: Thu Dec 10 15:33:15 2015 +0200
Committer: Sylvain Lebresne 
Committed: Mon Dec 14 10:57:20 2015 +0100

--
 CHANGES.txt  |  2 ++
 .../org/apache/cassandra/db/UnfilteredDeserializer.java  | 11 +--
 .../cassandra/db/compaction/CompactionManager.java   |  5 -
 3 files changed, 11 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/71bac92c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c30d35d..a0717c6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.2
+ * Fix upgrade data loss due to range tombstone deleting more data than then 
should
+   (CASSANDRA-10822)
 Merged from 2.2
  * Add property to allow listening on broadcast interface (CASSANDRA-9748)
  * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/71bac92c/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 66f6b71..bf9c2b8 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -406,7 +406,7 @@ public abstract class UnfilteredDeserializer
 
 public boolean hasNext()
 {
-// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below.
+// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below or the atom might be 
shadowed.
 while (next == null)
 {
 if (atoms.hasNext())
@@ -419,7 +419,8 @@ public abstract class UnfilteredDeserializer
 else
 {
 LegacyLayout.LegacyAtom atom = atoms.next();
-next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (!tombstoneTracker.isShadowed(atom))
+next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
 }
 }
 else if (tombstoneTracker.hasOpenTombstones())
@@ -498,7 +499,7 @@ public abstract class UnfilteredDeserializer
 if (isDone)
 return false;
 
-while (next == null)
+if (next == null)
 {
 next = readAtom();
 if (next == null)
@@ -506,9 +507,6 @@ public abstract class UnfilteredDeserializer
 isDone = true;
 return false;
 }
-
-if (tombstoneTracker.isShadowed(next))
-next = null;
 }
 return true;
 }
@@ -576,6 +574,7 @@ public abstract class UnfilteredDeserializer
  */
 public boolean isShadowed(LegacyLayout.LegacyAtom atom)
 {
+assert !hasClosingMarkerBefore(atom);
 long timestamp = atom.isCell() ? atom.asCell().timestamp : 
atom.asRangeTombstone().deletionTime.markedForDeleteAt();
 
 if (partitionDeletion.deletes(timestamp))

http://git-wip-us.apache.org/repos/asf/cassandra/blob/71bac92c/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index bd950e3..2234838 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -959,13 +959,16 @@ public class CompactionManager implements 
CompactionManagerMBean
  

[1/3] cassandra git commit: Make Stress compiles within eclipse

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 a8b9e533f -> 94e8d62cf


Make Stress compiles within eclipse

patch by Benjamin Lerer; reviewed by Jake Luciani for CASSANDRA-10807


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b30d657
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b30d657
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b30d657

Branch: refs/heads/cassandra-3.0
Commit: 9b30d6572fdb988796788ad4c7b8daabdef4e961
Parents: dff2214
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:17:50 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:22:30 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 96f2f4f..091ac52 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.13
+ * Make Stress compiles within eclipse (CASSANDRA-10807)
  * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
  * Allow cancellation of index summary redistribution (CASSANDRA-8805)
  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
index 00f866a..6b39d08 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
@@ -31,13 +31,13 @@ public abstract class Generator
 
 public final String name;
 public final AbstractType type;
-public final Class clazz;
+public final Class clazz;
 final long salt;
 final Distribution identityDistribution;
 final Distribution sizeDistribution;
 public final Distribution clusteringDistribution;
 
-public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
+public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
 {
 this.type = type;
 this.name = name;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
index bfa58ea..d82e01f 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
+++ b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
@@ -26,16 +26,17 @@ import java.util.List;
 
 import org.apache.cassandra.db.marshal.ListType;
 
-public class Lists extends Generator
+public class Lists extends Generator
 {
-final Generator valueType;
-final Object[] buffer;
+final Generator valueType;
+final T[] buffer;
 
-public Lists(String name, Generator valueType, GeneratorConfig config)
+@SuppressWarnings("unchecked")
+public Lists(String name, Generator valueType, GeneratorConfig config)
 {
 super(ListType.getInstance(valueType.type, true), config, name, 
List.class);
 this.valueType = valueType;
-buffer = new Object[(int) sizeDistribution.maxValue()];
+buffer = (T[]) new Object[(int) sizeDistribution.maxValue()];
 }
 
 public void setSeed(long seed)
@@ -45,7 +46,7 @@ public class Lists extends Generator
 }
 
 @Override
-public List generate()
+public List generate()
 {
 int size = (int) sizeDistribution.next();
 for (int i = 0 ; i < size ; i++)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
index 5c17b4e..b74d6b2 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
+++ 

[2/3] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/263763ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/263763ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/263763ab

Branch: refs/heads/cassandra-3.0
Commit: 263763ab9bfd9be902951d06f5d44f3a9d3f6f4f
Parents: 6ff7c9c 9b30d65
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:27:21 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:28:05 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/263763ab/CHANGES.txt
--
diff --cc CHANGES.txt
index d44a47b,091ac52..592ba0a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,12 +1,35 @@@
 -2.1.13
 +2.2.5
 + * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 + * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 + * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 + * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 + * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
 +Merged from 2.1:
+  * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
 - * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
   * Fix Stress profile parsing on Windows (CASSANDRA-10808)
  
 -
 -2.1.12
 +2.2.4
 + * Show CQL help in cqlsh in web browser (CASSANDRA-7225)
 + * Serialize on disk the proper SSTable compression ratio (CASSANDRA-10775)
 + * Reject index queries while the index is building (CASSANDRA-8505)
 + * CQL.textile syntax incorrectly includes optional keyspace for aggregate 
SFUNC and FINALFUNC (CASSANDRA-10747)
 + * Fix JSON update with prepared statements (CASSANDRA-10631)
 + * Don't do anticompaction after subrange repair (CASSANDRA-10422)
 + * Fix SimpleDateType type compatibility (CASSANDRA-10027)
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 + * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
 +Merged from 2.1:
   * Fix incremental repair hang when replica is down (CASSANDRA-10288)
   * Avoid writing range tombstones after END_OF_ROW marker (CASSANDRA-10791)
   * Optimize the way we check if a token is repaired in anticompaction 
(CASSANDRA-10768)

[3/3] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/94e8d62c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/94e8d62c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/94e8d62c

Branch: refs/heads/cassandra-3.0
Commit: 94e8d62cf8cea8918b01b038746be4b4e03af515
Parents: a8b9e53 263763a
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:29:07 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:29:17 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/94e8d62c/CHANGES.txt
--
diff --cc CHANGES.txt
index e20f5ed,592ba0a..c30d35d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -6,40 -5,12 +6,41 @@@ Merged from 2.
   * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
   * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
  Merged from 2.1:
+  * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
 - * Fix Stress profile parsing on Windows (CASSANDRA-10808)
  
 -2.2.4
 +3.0.1
 + * Avoid MV race during node decommission (CASSANDRA-10674)
 + * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 + * Handle single-column deletions correction in materialized views
 +   when the column is part of the view primary key (CASSANDRA-10796)
 + * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
 + * Fix bug with range tombstones on reverse queries and test coverage for
 +   AbstractBTreePartition (CASSANDRA-10059)
 + * Remove 64k limit on collection elements (CASSANDRA-10374)
 + * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
 + * Fix NPE on stream read error (CASSANDRA-10771)
 + * Normalize cqlsh DESC output (CASSANDRA-10431)
 + * Rejects partition range deletions when columns are specified 
(CASSANDRA-10739)
 + * Fix error when saving cached key for old format sstable (CASSANDRA-10778)
 + * Invalidate prepared statements on DROP INDEX (CASSANDRA-10758)
 + * Fix SELECT statement with IN restrictions on partition key,
 +   ORDER BY and LIMIT (CASSANDRA-10729)
 + * Improve stress performance over 1k threads (CASSANDRA-7217)
 + * Wait for migration responses to complete before bootstrapping 
(CASSANDRA-10731)
 + * Unable to create a function with argument of type Inet (CASSANDRA-10741)
 + * Fix backward incompatibiliy in CqlInputFormat (CASSANDRA-10717)
 + * Correctly preserve deletion info on updated rows when notifying indexers
 +   of single-row deletions (CASSANDRA-10694)
 + * Notify indexers of partition delete during cleanup (CASSANDRA-10685)
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
 + * Fix regression on split size in CqlInputFormat (CASSANDRA-10835)
 + * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 + * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
 + * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
   * Show CQL help in cqlsh in web browser (CASSANDRA-7225)
   * Serialize on disk the proper SSTable compression ratio (CASSANDRA-10775)
   * Reject index queries while the index is building (CASSANDRA-8505)

[jira] [Comment Edited] (CASSANDRA-9179) Unable to "point in time" restore if table/cf has been recreated

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055708#comment-15055708
 ] 

Marcus Eriksson edited comment on CASSANDRA-9179 at 12/14/15 9:45 AM:
--

in general, +1 on the patches

The 2.2 patch seems to be based on 2.1, should this go into 2.2+?

nits:
* on the 2.1/2.2 patch, whitespace on line 88 in CFPropDefs
* in 3.0, should the KW_ID go into TableParams.Option ?


was (Author: krummas):
+1 on the patches

The 2.2 patch seems to be based on 2.1, should this go into 2.2+?

and a small nit on the 2.1/2.2 patch, whitespace on line 88 in CFPropDefs

> Unable to "point in time" restore if table/cf has been recreated
> 
>
> Key: CASSANDRA-9179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9179
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Distributed Metadata
>Reporter: Jon Moses
>Assignee: Branimir Lambov
>  Labels: doc-impacting
>
> With Cassandra 2.1, and the addition of the CF UUID, the ability to do a 
> "point in time" restore by restoring a snapshot and replaying commitlogs is 
> lost if the table has been dropped and recreated.
> When the table is recreated, the cf_id changes, and the commitlog replay 
> mechanism skips the desired mutations as the cf_id no longer matches what's 
> present in the schema.
> There should exist a way to inform the replay that you want the mutations 
> replayed even if the cf_id doesn't match.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Fix missing row data after range tombstone in legacy data

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9bfe61357 -> 6bc363726


Fix missing row data after range tombstone in legacy data

patch by blambov; reviewed by slebresne for CASSANDRA-10822


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/71bac92c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/71bac92c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/71bac92c

Branch: refs/heads/trunk
Commit: 71bac92cf1df08de194a8283f382a7950b3a78ed
Parents: 94e8d62
Author: Branimir Lambov 
Authored: Thu Dec 10 15:33:15 2015 +0200
Committer: Sylvain Lebresne 
Committed: Mon Dec 14 10:57:20 2015 +0100

--
 CHANGES.txt  |  2 ++
 .../org/apache/cassandra/db/UnfilteredDeserializer.java  | 11 +--
 .../cassandra/db/compaction/CompactionManager.java   |  5 -
 3 files changed, 11 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/71bac92c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c30d35d..a0717c6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.2
+ * Fix upgrade data loss due to range tombstone deleting more data than then 
should
+   (CASSANDRA-10822)
 Merged from 2.2
  * Add property to allow listening on broadcast interface (CASSANDRA-9748)
  * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/71bac92c/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 66f6b71..bf9c2b8 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -406,7 +406,7 @@ public abstract class UnfilteredDeserializer
 
 public boolean hasNext()
 {
-// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below.
+// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below or the atom might be 
shadowed.
 while (next == null)
 {
 if (atoms.hasNext())
@@ -419,7 +419,8 @@ public abstract class UnfilteredDeserializer
 else
 {
 LegacyLayout.LegacyAtom atom = atoms.next();
-next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (!tombstoneTracker.isShadowed(atom))
+next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
 }
 }
 else if (tombstoneTracker.hasOpenTombstones())
@@ -498,7 +499,7 @@ public abstract class UnfilteredDeserializer
 if (isDone)
 return false;
 
-while (next == null)
+if (next == null)
 {
 next = readAtom();
 if (next == null)
@@ -506,9 +507,6 @@ public abstract class UnfilteredDeserializer
 isDone = true;
 return false;
 }
-
-if (tombstoneTracker.isShadowed(next))
-next = null;
 }
 return true;
 }
@@ -576,6 +574,7 @@ public abstract class UnfilteredDeserializer
  */
 public boolean isShadowed(LegacyLayout.LegacyAtom atom)
 {
+assert !hasClosingMarkerBefore(atom);
 long timestamp = atom.isCell() ? atom.asCell().timestamp : 
atom.asRangeTombstone().deletionTime.markedForDeleteAt();
 
 if (partitionDeletion.deletes(timestamp))

http://git-wip-us.apache.org/repos/asf/cassandra/blob/71bac92c/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index bd950e3..2234838 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -959,13 +959,16 @@ public class CompactionManager implements 
CompactionManagerMBean
  

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[slebresne]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6bc36372
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6bc36372
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6bc36372

Branch: refs/heads/trunk
Commit: 6bc3637260f287a031f2ea8a7e155a6b7692fe92
Parents: 9bfe613 71bac92
Author: Sylvain Lebresne 
Authored: Mon Dec 14 10:58:53 2015 +0100
Committer: Sylvain Lebresne 
Committed: Mon Dec 14 10:58:53 2015 +0100

--
 CHANGES.txt  |  5 -
 .../org/apache/cassandra/db/UnfilteredDeserializer.java  | 11 +--
 .../cassandra/db/compaction/CompactionManager.java   |  5 -
 3 files changed, 13 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6bc36372/CHANGES.txt
--
diff --cc CHANGES.txt
index 0ca5277,a0717c6..7d0ca2e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,22 -1,12 +1,25 @@@
 -3.0.2
 +3.2
 + * Group pending compactions based on table (CASSANDRA-10718)
 + * Add compressor name in sstablemetadata output (CASSANDRA-9879)
 + * Fix type casting for counter columns (CASSANDRA-10824)
 + * Prevent running Cassandra as root (CASSANDRA-8142)
 + * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
- Merged from 2.2:
++Merged from 3.0:
+  * Fix upgrade data loss due to range tombstone deleting more data than then 
should
+(CASSANDRA-10822)
+ Merged from 2.2
   * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 - * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 - * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 - * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 - * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
  Merged from 2.1:
   * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6bc36372/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 7a2e71f,2234838..fafab69
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -965,8 -968,7 +968,8 @@@ public class CompactionManager implemen
  expectedBloomFilterSize,
  repairedAt,
  sstable.getSSTableLevel(),
- sstable.header,
+ header,
 +cfs.indexManager.listIndexes(),
  txn);
  }
  

cassandra git commit: Make Stress compiles within eclipse

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 dff221459 -> 9b30d6572


Make Stress compiles within eclipse

patch by Benjamin Lerer; reviewed by Jake Luciani for CASSANDRA-10807


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b30d657
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b30d657
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b30d657

Branch: refs/heads/cassandra-2.1
Commit: 9b30d6572fdb988796788ad4c7b8daabdef4e961
Parents: dff2214
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:17:50 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:22:30 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 96f2f4f..091ac52 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.13
+ * Make Stress compiles within eclipse (CASSANDRA-10807)
  * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
  * Allow cancellation of index summary redistribution (CASSANDRA-8805)
  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
index 00f866a..6b39d08 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
@@ -31,13 +31,13 @@ public abstract class Generator
 
 public final String name;
 public final AbstractType type;
-public final Class clazz;
+public final Class clazz;
 final long salt;
 final Distribution identityDistribution;
 final Distribution sizeDistribution;
 public final Distribution clusteringDistribution;
 
-public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
+public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
 {
 this.type = type;
 this.name = name;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
index bfa58ea..d82e01f 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
+++ b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
@@ -26,16 +26,17 @@ import java.util.List;
 
 import org.apache.cassandra.db.marshal.ListType;
 
-public class Lists extends Generator
+public class Lists extends Generator
 {
-final Generator valueType;
-final Object[] buffer;
+final Generator valueType;
+final T[] buffer;
 
-public Lists(String name, Generator valueType, GeneratorConfig config)
+@SuppressWarnings("unchecked")
+public Lists(String name, Generator valueType, GeneratorConfig config)
 {
 super(ListType.getInstance(valueType.type, true), config, name, 
List.class);
 this.valueType = valueType;
-buffer = new Object[(int) sizeDistribution.maxValue()];
+buffer = (T[]) new Object[(int) sizeDistribution.maxValue()];
 }
 
 public void setSeed(long seed)
@@ -45,7 +46,7 @@ public class Lists extends Generator
 }
 
 @Override
-public List generate()
+public List generate()
 {
 int size = (int) sizeDistribution.next();
 for (int i = 0 ; i < size ; i++)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
index 5c17b4e..b74d6b2 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
+++ 

[2/4] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/263763ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/263763ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/263763ab

Branch: refs/heads/trunk
Commit: 263763ab9bfd9be902951d06f5d44f3a9d3f6f4f
Parents: 6ff7c9c 9b30d65
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:27:21 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:28:05 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/263763ab/CHANGES.txt
--
diff --cc CHANGES.txt
index d44a47b,091ac52..592ba0a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,12 +1,35 @@@
 -2.1.13
 +2.2.5
 + * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 + * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 + * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 + * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 + * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
 +Merged from 2.1:
+  * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
 - * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
   * Fix Stress profile parsing on Windows (CASSANDRA-10808)
  
 -
 -2.1.12
 +2.2.4
 + * Show CQL help in cqlsh in web browser (CASSANDRA-7225)
 + * Serialize on disk the proper SSTable compression ratio (CASSANDRA-10775)
 + * Reject index queries while the index is building (CASSANDRA-8505)
 + * CQL.textile syntax incorrectly includes optional keyspace for aggregate 
SFUNC and FINALFUNC (CASSANDRA-10747)
 + * Fix JSON update with prepared statements (CASSANDRA-10631)
 + * Don't do anticompaction after subrange repair (CASSANDRA-10422)
 + * Fix SimpleDateType type compatibility (CASSANDRA-10027)
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 + * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
 +Merged from 2.1:
   * Fix incremental repair hang when replica is down (CASSANDRA-10288)
   * Avoid writing range tombstones after END_OF_ROW marker (CASSANDRA-10791)
   * Optimize the way we check if a token is repaired in anticompaction 
(CASSANDRA-10768)

[1/4] cassandra git commit: Make Stress compiles within eclipse

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk a808769fc -> 9bfe61357


Make Stress compiles within eclipse

patch by Benjamin Lerer; reviewed by Jake Luciani for CASSANDRA-10807


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b30d657
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b30d657
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b30d657

Branch: refs/heads/trunk
Commit: 9b30d6572fdb988796788ad4c7b8daabdef4e961
Parents: dff2214
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:17:50 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:22:30 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 96f2f4f..091ac52 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.13
+ * Make Stress compiles within eclipse (CASSANDRA-10807)
  * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
  * Allow cancellation of index summary redistribution (CASSANDRA-8805)
  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
index 00f866a..6b39d08 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Generator.java
@@ -31,13 +31,13 @@ public abstract class Generator
 
 public final String name;
 public final AbstractType type;
-public final Class clazz;
+public final Class clazz;
 final long salt;
 final Distribution identityDistribution;
 final Distribution sizeDistribution;
 public final Distribution clusteringDistribution;
 
-public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
+public Generator(AbstractType type, GeneratorConfig config, String 
name, Class clazz)
 {
 this.type = type;
 this.name = name;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
index bfa58ea..d82e01f 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
+++ b/tools/stress/src/org/apache/cassandra/stress/generate/values/Lists.java
@@ -26,16 +26,17 @@ import java.util.List;
 
 import org.apache.cassandra.db.marshal.ListType;
 
-public class Lists extends Generator
+public class Lists extends Generator
 {
-final Generator valueType;
-final Object[] buffer;
+final Generator valueType;
+final T[] buffer;
 
-public Lists(String name, Generator valueType, GeneratorConfig config)
+@SuppressWarnings("unchecked")
+public Lists(String name, Generator valueType, GeneratorConfig config)
 {
 super(ListType.getInstance(valueType.type, true), config, name, 
List.class);
 this.valueType = valueType;
-buffer = new Object[(int) sizeDistribution.maxValue()];
+buffer = (T[]) new Object[(int) sizeDistribution.maxValue()];
 }
 
 public void setSeed(long seed)
@@ -45,7 +46,7 @@ public class Lists extends Generator
 }
 
 @Override
-public List generate()
+public List generate()
 {
 int size = (int) sizeDistribution.next();
 for (int i = 0 ; i < size ; i++)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b30d657/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
index 5c17b4e..b74d6b2 100644
--- a/tools/stress/src/org/apache/cassandra/stress/generate/values/Sets.java
+++ 

[jira] [Commented] (CASSANDRA-9179) Unable to "point in time" restore if table/cf has been recreated

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055844#comment-15055844
 ] 

Branimir Lambov commented on CASSANDRA-9179:


bq. in 3.0, should the KW_ID go into TableParams.Option

Not sure. I did not want to put it in the {{TableParams}} as they can be 
changed using {{CFMetaData.params}}. If it's not a {{TableParams}} value I 
didn't think it's a good idea to put the identifier (and validation etc.) there.

I targeted 2.2, but it looks like the patch is fine on 2.1 as well. Uploaded a 
2.1 branch with the whitespace removed:
|[2.1/2.2 
code|https://github.com/blambov/cassandra/tree/9179-with-id-2.1]|[utests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-9179-with-id-2.1-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-9179-with-id-2.1-dtest/]|

> Unable to "point in time" restore if table/cf has been recreated
> 
>
> Key: CASSANDRA-9179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9179
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Distributed Metadata
>Reporter: Jon Moses
>Assignee: Branimir Lambov
>  Labels: doc-impacting
>
> With Cassandra 2.1, and the addition of the CF UUID, the ability to do a 
> "point in time" restore by restoring a snapshot and replaying commitlogs is 
> lost if the table has been dropped and recreated.
> When the table is recreated, the cf_id changes, and the commitlog replay 
> mechanism skips the desired mutations as the cf_id no longer matches what's 
> present in the schema.
> There should exist a way to inform the replay that you want the mutations 
> replayed even if the cf_id doesn't match.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9179) Unable to "point in time" restore if table/cf has been recreated

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055889#comment-15055889
 ] 

Marcus Eriksson commented on CASSANDRA-9179:


bq. I did not want to put it in the TableParams as they can be changed using 
CFMetaData.params. If it's not a TableParams value I didn't think it's a good 
idea to put the identifier (and validation etc.) there.
ok, +1

I'll commit once the 2.1 dtests pass

> Unable to "point in time" restore if table/cf has been recreated
> 
>
> Key: CASSANDRA-9179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9179
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Distributed Metadata
>Reporter: Jon Moses
>Assignee: Branimir Lambov
>  Labels: doc-impacting
>
> With Cassandra 2.1, and the addition of the CF UUID, the ability to do a 
> "point in time" restore by restoring a snapshot and replaying commitlogs is 
> lost if the table has been dropped and recreated.
> When the table is recreated, the cf_id changes, and the commitlog replay 
> mechanism skips the desired mutations as the cf_id no longer matches what's 
> present in the schema.
> There should exist a way to inform the replay that you want the mutations 
> replayed even if the cf_id doesn't match.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9179) Unable to "point in time" restore if table/cf has been recreated

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-9179:
---
Fix Version/s: 3.x
   3.0.x
   2.2.x
   2.1.x

> Unable to "point in time" restore if table/cf has been recreated
> 
>
> Key: CASSANDRA-9179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9179
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Distributed Metadata
>Reporter: Jon Moses
>Assignee: Branimir Lambov
>  Labels: doc-impacting
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> With Cassandra 2.1, and the addition of the CF UUID, the ability to do a 
> "point in time" restore by restoring a snapshot and replaying commitlogs is 
> lost if the table has been dropped and recreated.
> When the table is recreated, the cf_id changes, and the commitlog replay 
> mechanism skips the desired mutations as the cf_id no longer matches what's 
> present in the schema.
> There should exist a way to inform the replay that you want the mutations 
> replayed even if the cf_id doesn't match.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10871) MemtableFlushWriter blocks and no flushing happens

[Rafael Harutyunyan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rafael Harutyunyan updated CASSANDRA-10871:
---
Environment: 
Linux cassandra1 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 13 22:55:16 UTC 2015 
x86_64 x86_64 x86_64 GNU/Linux; Java(TM) SE Runtime Environment (build 
1.7.0_67-b01)


  was:Linux cassandra1 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 13 22:55:16 UTC 
2015 x86_64 x86_64 x86_64 GNU/Linux


> MemtableFlushWriter blocks and no flushing happens
> --
>
> Key: CASSANDRA-10871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10871
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Local Write-Read Paths
> Environment: Linux cassandra1 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu 
> Aug 13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux; Java(TM) SE Runtime 
> Environment (build 1.7.0_67-b01)
>Reporter: Rafael Harutyunyan
>Priority: Critical
> Fix For: 2.1.11
>
> Attachments: full_thread_dump.txt
>
>
> After some time MemtableFlushWriter thread blocks, resulting first full 
> filling of the FlushWriterQueue, than full filling of MutationStage queue. 
> After this 2 things might happen - Cassandra might drop the queued mutations 
> and everything becomes normal or it shuts down with insufficient HeapSpace.
> Here is the thread dump.
> {noformat}
> "MemtableFlushWriter:3" - Thread t@2610
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock  (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:51" t@2638
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1502)
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
>   - locked <7ef8cd1b> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "MemtableFlushWriter:4" - Thread t@2616
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock  (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:51" t@2638
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1502)
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
>   - locked <2f842d9b> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {noformat}
> and here are the tpsats
> {noformat}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> CounterMutationStage  0 0  0 0
>  0
> ReadStage 0 0 28 0
>  0
> RequestResponseStage  0 02020253 0
>  0
> MutationStage  

[jira] [Commented] (CASSANDRA-9179) Unable to "point in time" restore if table/cf has been recreated

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055708#comment-15055708
 ] 

Marcus Eriksson commented on CASSANDRA-9179:


+1 on the patches

The 2.2 patch seems to be based on 2.1, should this go into 2.2+?

and a small nit on the 2.1/2.2 patch, whitespace on line 88 in CFPropDefs

> Unable to "point in time" restore if table/cf has been recreated
> 
>
> Key: CASSANDRA-9179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9179
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Distributed Metadata
>Reporter: Jon Moses
>Assignee: Branimir Lambov
>  Labels: doc-impacting
>
> With Cassandra 2.1, and the addition of the CF UUID, the ability to do a 
> "point in time" restore by restoring a snapshot and replaying commitlogs is 
> lost if the table has been dropped and recreated.
> When the table is recreated, the cf_id changes, and the commitlog replay 
> mechanism skips the desired mutations as the cf_id no longer matches what's 
> present in the schema.
> There should exist a way to inform the replay that you want the mutations 
> replayed even if the cf_id doesn't match.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9472) Reintroduce off heap memtables

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056156#comment-15056156
 ] 

Stefania commented on CASSANDRA-9472:
-

I've finally have a reworked patch and the first stable CI results:

|[patch|https://github.com/stef1927/cassandra/commits/9472]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9472-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9472-dtest/]|

I still have 3 dtests and 2 utests to check but the core issues should be fixed 
now. 

The initial patch was correct but there were three more problems to address, 
the most significant of which is by far the last one:

* Segmentation faults: the instances with {{peer = 0}} for {{NativeCell}} and 
{{NativeClustering}} used to measure empty object sizes were causing 
segmentation faults, at least in the debugger, because of base class methods 
such as {{toString()}} calling methods using a null {{peer}}. I've added guards 
for {{peer == 0}} in all methods but perhaps we ought to come up with a better 
way to estimate empty object sizes.

* Wrong byte ordering: by default hollow byte buffers were given native byte 
ordering (little endian on my box) but if the original byte buffers were big 
endian then the correct order is actually big endian. Since we only deal with 
big endian buffers, to the best of my knowledge, instead of wasting space and 
time storing the initial encoding of each byte buffer I've added assertions and 
enforced big endian ordering when creating hollow byte buffers.

* Released native memory: this was by far the hardest thing to debug and fix. 
After a memtable is flushed, the native memory is released and the native 
objects, or the byte buffers returned by them, are no longer valid. The 
existing mechanism of copying such objects on the heap had become inadequate 
due to the refactoring of 8099. The strategy I've adopted is to copy any native 
row or decorated key that is returned by {{AbstractBTreePartition}} or its row 
iterators. As far as I understood the code, this should be sufficient provided 
the partition and row iterators are guarded by a read op barrier normally via 
an execution controller.

> Reintroduce off heap memtables
> --
>
> Key: CASSANDRA-9472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9472
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Stefania
> Fix For: 3.2
>
>
> CASSANDRA-8099 removes off heap memtables. We should reintroduce them ASAP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10750) Minor code improvements

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056159#comment-15056159
 ] 

Stefania commented on CASSANDRA-10750:
--

[~snazy] : I have just started working on the review and I hope to have some 
more comments by tomorrow. In the meantime you may want to rebase and take a 
look at the cassci results as they don't look too good. Aside from the usual 
dtests time outs, there seems to be some NPEs. The utests also had a bad run, 
at a minimum run them one more time to make sure the additional time outs are 
unrelated.

> Minor code improvements
> ---
>
> Key: CASSANDRA-10750
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10750
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
>
> Went though several IDE inspections and found some places in the code that 
> could be improved. These are just minor improvements and no bug fixes (except 
> one minor "theoretical" thing).
> The [branch on github against 
> trunk|https://github.com/snazy/cassandra/tree/10750-code-opts-trunk] contains 
> a series of commits:
> * simplify Mutation.apply to remove the casts
> * "minor code improvements" just replaces some expressions that are 
> effectively constant
> * remove unused assignments (probably just cosmetic)
> * collapse identical if-branches (probably just cosmetic)
> * empty array constants
> * fix printf usage (could potentially raise an exception in printf)
> * replace tail-recursion in some critical sections (as the JVM cannot 
> optimize that AFAIK)
> * remove methods identical to their super methods (probably just cosmetic)
> [cassci results 
> here|http://cassci.datastax.com/view/Dev/view/snazy/search/?q=snazy-10750-]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer

[Mike Adamson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056182#comment-15056182
 ] 

Mike Adamson commented on CASSANDRA-10852:
--

Good point. I've updated the branch to use a default implementation of the 
method.

> Add requireAuthorization method to IAuthorizer
> --
>
> Key: CASSANDRA-10852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10852
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.2
>
>
> The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate 
> whether an implementation requires an explicit login. For consistency it 
> would be useful for 3rd party implementers if the {{IAuthorizer}} had a 
> similar method {{requireAuthorization}} that would indicate if the 
> implementation required explicit authorization.
> This would mean that we could remove and explicit {{instanceof}} checks for 
> {{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-9258:
-
Attachment: 0001-pending-ranges-map.patch

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges-map.patch
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Dikang Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057349#comment-15057349
 ] 

Dikang Gu commented on CASSANDRA-9258:
--

[~blambov], thanks for the review, addressed the comments.

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-9258:
-
Attachment: (was: 0001-pending-ranges-map.patch)

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10099) Improve concurrency in CompactionStrategyManager

[Rafael Harutyunyan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057449#comment-15057449
 ] 

Rafael Harutyunyan commented on CASSANDRA-10099:


Probably CASSANDRA-10871 is related to this?

> Improve concurrency in CompactionStrategyManager
> 
>
> Key: CASSANDRA-10099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10099
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Yuki Morishita
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.x
>
>
> Continue discussion from CASSANDRA-9882.
> CompactionStrategyManager(WrappingCompactionStrategy for <3.0) tracks SSTable 
> changes mainly for separating repaired / unrepaired SSTables (+ LCS manages 
> level).
> This is blocking operation, and can lead to block of flush etc. when 
> determining next background task takes longer.
> Explore the way to mitigate this concurrency issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10860) repair anticompaction tests are failing on 2.2+

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-10860:
---

Assignee: Marcus Eriksson

> repair anticompaction tests are failing on 2.2+
> ---
>
> Key: CASSANDRA-10860
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10860
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Compaction, Testing
>Reporter: Philip Thompson
>Assignee: Marcus Eriksson
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> {{repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test}} 
> and {{repair_test.TestRepair.no_anticompaction_after_dclocal_repair_test}} 
> are both failing on 2.2 and 3.0, with run on clusters not using vnodes. You 
> can run the tests by setting DISABLE_VNODES=true.
> These tests were added for CASSANDRA-10422.
> When I run these locally, it appears from the logs that the repair is not 
> running on every node, despite being specified in the nodetool command. See 
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/168/testReport/ 
> for an example failure. I can reproduce this consistently locally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10871) MemtableFlushWriter blocks and no flushing happens

[Rafael Harutyunyan (JIRA)]

Rafael Harutyunyan created CASSANDRA-10871:
--

 Summary: MemtableFlushWriter blocks and no flushing happens
 Key: CASSANDRA-10871
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10871
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction, Local Write-Read Paths
 Environment: Linux cassandra1 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 
13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Rafael Harutyunyan
Priority: Critical
 Fix For: 2.1.11
 Attachments: full_thread_dump.txt

After some time MemtableFlushWriter thread blocks, resulting first full filling 
of the FlushWriterQueue, than full filling of MutationStage queue. After this 2 
things might happen - Cassandra might drop the queued mutations and everything 
becomes normal or it shuts down with insufficient HeapSpace.
Here is the thread dump.
{noformat}

"MemtableFlushWriter:3" - Thread t@2610
   java.lang.Thread.State: BLOCKED
at 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
- waiting to lock  (a 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
"CompactionExecutor:51" t@2638
at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
at 
org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
at 
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
at 
org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1502)
at 
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
- locked <7ef8cd1b> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"MemtableFlushWriter:4" - Thread t@2616
   java.lang.Thread.State: BLOCKED
at 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
- waiting to lock  (a 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
"CompactionExecutor:51" t@2638
at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
at 
org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
at 
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
at 
org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1502)
at 
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
- locked <2f842d9b> (a java.util.concurrent.ThreadPoolExecutor$Worker)
{noformat}

and here are the tpsats
{noformat}
Pool NameActive   Pending  Completed   Blocked  All 
time blocked
CounterMutationStage  0 0  0 0  
   0
ReadStage 0 0 28 0  
   0
RequestResponseStage  0 02020253 0  
   0
MutationStage32 63221   27858588 0  
   0
ReadRepairStage   0 0  0 0  
   0
GossipStage   0 0  16430 0  
   0
CacheCleanupExecutor  0 0  0 0  
   0
AntiEntropyStage  0 0   3008 0  
   0
MigrationStage0 0  0 0  
   0
Sampler   0 0  0 0  
   0
ValidationExecutor 

[jira] [Commented] (CASSANDRA-10871) MemtableFlushWriter blocks and no flushing happens

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057489#comment-15057489
 ] 

Marcus Eriksson commented on CASSANDRA-10871:
-

How did you end up in the situation with that many sstables? Incremental repair?

> MemtableFlushWriter blocks and no flushing happens
> --
>
> Key: CASSANDRA-10871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10871
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Local Write-Read Paths
> Environment: Linux cassandra1 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu 
> Aug 13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux; Java(TM) SE Runtime 
> Environment (build 1.7.0_67-b01)
>Reporter: Rafael Harutyunyan
>Priority: Critical
> Fix For: 2.1.11
>
> Attachments: full_thread_dump.txt
>
>
> After some time MemtableFlushWriter thread blocks, resulting first full 
> filling of the FlushWriterQueue, than full filling of MutationStage queue. 
> After this 2 things might happen - Cassandra might drop the queued mutations 
> and everything becomes normal or it shuts down with insufficient HeapSpace.
> Here is the thread dump.
> {noformat}
> "MemtableFlushWriter:3" - Thread t@2610
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock  (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:51" t@2638
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1502)
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
>   - locked <7ef8cd1b> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "MemtableFlushWriter:4" - Thread t@2616
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock  (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:51" t@2638
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1502)
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
>   - locked <2f842d9b> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {noformat}
> and here are the tpsats
> {noformat}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> CounterMutationStage  0 0  0 0
>  0
> ReadStage 0 0 28 0
>  0
> RequestResponseStage  0 02020253 0
>  0
> MutationStage32 63221   27858588 0
>  0
> ReadRepairStage   0 0  0 0
>  0
> GossipStage

[jira] [Commented] (CASSANDRA-10852) Add requireAuthorization method to IAuthorizer

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056036#comment-15056036
 ] 

Aleksey Yeschenko commented on CASSANDRA-10852:
---

Not a review, however:
1. You are not allowed to add new abstract interface methods to public APIs. 
For 3.x, though, as we are forcing Java 8, you can use default methods.
2. {{@Override}} missing in the implementations once you switch to {{default}}

> Add requireAuthorization method to IAuthorizer
> --
>
> Key: CASSANDRA-10852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10852
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.2
>
>
> The {{IAuthenticator}} interface has a {{requireAuthentication}} to indicate 
> whether an implementation requires an explicit login. For consistency it 
> would be useful for 3rd party implementers if the {{IAuthorizer}} had a 
> similar method {{requireAuthorization}} that would indicate if the 
> implementation required explicit authorization.
> This would mean that we could remove and explicit {{instanceof}} checks for 
> {{AllowAllAuthenticator}} and {{AllowAllAuthorizer}} in the code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10735) Support netty openssl (netty-tcnative) for client encryption

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056061#comment-15056061
 ] 

Aleksey Yeschenko commented on CASSANDRA-10735:
---

[~andrew.tolbert] thanks for the update

> Support netty openssl (netty-tcnative) for client encryption
> 
>
> Key: CASSANDRA-10735
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10735
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Andy Tolbert
> Fix For: 3.x
>
> Attachments: nettyssl-bench.tgz, nettysslbench.png, 
> nettysslbench_small.png
>
>
> The java-driver recently added support for using netty openssl via 
> [netty-tcnative|http://netty.io/wiki/forked-tomcat-native.html] in 
> [JAVA-841|https://datastax-oss.atlassian.net/browse/JAVA-841], this shows a 
> very measured improvement (numbers incoming on that ticket).   It seems 
> likely that this can offer improvement if implemented C* side as well.
> Since netty-tcnative has platform specific requirements, this should not be 
> made the default, but rather be an option that one can use.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10541) cqlshlib tests cannot run on Windows

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056063#comment-15056063
 ] 

Jim Witschey commented on CASSANDRA-10541:
--

bq. Can we mark as ready to commit then?

Done.

> cqlshlib tests cannot run on Windows
> 
>
> Key: CASSANDRA-10541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: cqlsh, windows
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> --
> Traceback (most recent call last):
>   File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in 
> loadTestsFromName
> addr.filename, addr.module)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in 
> importFromPath
> return self.importFromDir(dir_path, fqname)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in 
> importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
>   File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in 
> from .cassconnect import create_test_db, remove_test_db
>   File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in 
> from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
>   File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in 
> os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no support for symlinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10711) NoSuchElementException when executing empty batch.

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056044#comment-15056044
 ] 

Sylvain Lebresne commented on CASSANDRA-10711:
--

[~thobbs] It doesn't seem you've actually started the tests listed above (the 
patch lgtm otherwise).

> NoSuchElementException when executing empty batch.
> --
>
> Key: CASSANDRA-10711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10711
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.0, OSS 42.1
>Reporter: Jaroslav Kamenik
>Assignee: ZhaoYang
>  Labels: triaged
> Fix For: 3.0.x
>
> Attachments: CASSANDRA-10711-trunk.patch
>
>
> After upgrade to C* 3.0, it fails when executes empty batch:
> {code}
> java.util.NoSuchElementException: null
> at java.util.ArrayList$Itr.next(ArrayList.java:854) ~[na:1.8.0_60]
> at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:737)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:356)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:337)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:323)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processBatch(QueryProcessor.java:490)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processBatch(QueryProcessor.java:480)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.transport.messages.BatchMessage.execute(BatchMessage.java:217)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.0.jar:3.0.0]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.0.jar:3.0.0]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_60]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [apache-cassandra-3.0.0.jar:3.0.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.0.0.jar:3.0.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056001#comment-15056001
 ] 

Aleksey Yeschenko commented on CASSANDRA-9318:
--

CASSANDRA-9834 relevant here (and related discussion in CASSANDRA-6230).

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9302) Optimize cqlsh COPY FROM, part 3

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055963#comment-15055963
 ] 

Stefania commented on CASSANDRA-9302:
-

I've rebased since CASSANDRA-10799 renamed _copy.py_ to _copyutil.py_.

> Optimize cqlsh COPY FROM, part 3
> 
>
> Key: CASSANDRA-9302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9302
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> We've had some discussion moving to Spark CSV import for bulk load in 3.x, 
> but people need a good bulk load tool now.  One option is to add a separate 
> Java bulk load tool (CASSANDRA-9048), but if we can match that performance 
> from cqlsh I would prefer to leave COPY FROM as the preferred option to which 
> we point people, rather than adding more tools that need to be supported 
> indefinitely.
> Previous work on COPY FROM optimization was done in CASSANDRA-7405 and 
> CASSANDRA-8225.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056216#comment-15056216
 ] 

Branimir Lambov commented on CASSANDRA-9258:


The solution looks good overall.

{{PendingRangeMaps}} could be declared iterable so that the for-in syntax can 
be used in e.g. {{TokenMetadata.getPendingRangesMM}}.

The comparators should not be created by the constructor as they can just as 
well be static final constants. It is not very clear why you need 4 of them, a 
comment explaining what each ordering does would help a lot (e.g. sorts ends 
ascending, and its secondary comparison for starts is chosen in such a way that 
and  ranges come before non-wraparound intervals with the same end).

{{PendingRangeMaps.addPendingRange}} needs a comment on why adding the address 
is done only once in the found case. Or restate it as finding the list to add 
to, creating if necessary, then adding the new address, for example:
{code}
List addresses = ascending.get(range);
if (addresses == null)
{
addresses = new ArrayList(1);
ascending.put(range, addresses);
descending.put(range, addresses);
}
addresses.add(address);
{code}

In {{PendingRangeMaps.pendingEndpointsFor}} the {{keySet, containsKey, get}} 
sequence is a bit more expensive than it could be: either use {{entrySet}} to 
avoid {{smaller.get}}, or {{bigger.get}} instead of {{containsKey}} to directly 
use the value if it's not null. Similarly, the two wrap-around loops should be 
on the {{entrySet}} s to avoid unnecessarily searching for the entry again.


> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges-map.patch
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X

[Aleksey Yeschenko (JIRA)]

Aleksey Yeschenko created CASSANDRA-10857:
-

 Summary: Allow dropping COMPACT STORAGE flag from tables in 3.X
 Key: CASSANDRA-10857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
 Project: Cassandra
  Issue Type: Improvement
  Components: CQL, Distributed Metadata
Reporter: Aleksey Yeschenko
 Fix For: 3.x


Thrift allows users to define flexible mixed column families - where certain 
columns would have explicitly pre-defined names, potentially non-default 
validation types, and be indexed.

Example:
{code}
create column family foo
and default_validation_class = UTF8Type
and column_metadata = [
{column_name: bar, validation_class: Int32Type, index_type: KEYS},
{column_name: baz, validation_class: UUIDType, index_type: KEYS}
];
{code}

Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and 
{{UUIDType}}, respectively, and be indexed. Columns with any other name will be 
validated by {{UTF8Type}} and will not be indexed.

With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns 
internally. However, being {{WITH COMPACT STORAGE}}, the table will only expose 
{{bar}} and {{baz}} columns. Accessing any dynamic columns (any column not 
named {{bar}} and {{baz}}) right now requires going through Thrift.

This is blocking Thrift -> CQL migration for users who have mixed 
dynamic/static column families. That said, it *shouldn't* be hard to allow 
users to drop the {{compact}} flag to expose the table as it is internally now, 
and be able to access all columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: bump versions for 3.1.1

[jake]

bump versions for 3.1.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8347d377
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8347d377
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8347d377

Branch: refs/heads/cassandra-3.1.1
Commit: 8347d37716d318956591ab7d5688774a083e5bfb
Parents: feaed2a
Author: T Jake Luciani 
Authored: Mon Dec 14 12:29:25 2015 -0500
Committer: T Jake Luciani 
Committed: Mon Dec 14 12:29:25 2015 -0500

--
 NEWS.txt | 9 +
 build.xml| 2 +-
 debian/changelog | 6 ++
 3 files changed, 16 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8347d377/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index e296544..7472e13 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,15 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+3.1.1
+=
+
+General
+---
+   - This release contains a critical fix regarding data loss when upgrading
+ a table with row tombstones.  
https://issues.apache.org/jira/browse/CASSANDRA-10822
+   
+
 3.1
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8347d377/build.xml
--
diff --git a/build.xml b/build.xml
index 731c26e..333cbcd 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8347d377/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 0714269..c3b7f0d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.1.1) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 14 Dec 2015 12:27:31 -0500
+
 cassandra (3.1) unstable; urgency=medium
 
   * New release

[1/2] cassandra git commit: Fix missing row data after range tombstone in legacy data

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.1.1 [created] 8347d3771


Fix missing row data after range tombstone in legacy data

patch by blambov; reviewed by slebresne for CASSANDRA-10822

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/feaed2a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/feaed2a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/feaed2a4

Branch: refs/heads/cassandra-3.1.1
Commit: feaed2a4362500f2b79fc90a728b50656338af00
Parents: e092873
Author: Branimir Lambov 
Authored: Thu Dec 10 15:33:15 2015 +0200
Committer: T Jake Luciani 
Committed: Mon Dec 14 12:19:17 2015 -0500

--
 CHANGES.txt  |  4 
 .../org/apache/cassandra/db/UnfilteredDeserializer.java  | 11 +--
 .../cassandra/db/compaction/CompactionManager.java   |  5 -
 3 files changed, 13 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/feaed2a4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a2e9434..dce2a06 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+3.1.1
+ * Fix upgrade data loss due to range tombstone deleting more data than then 
should
+   (CASSANDRA-10822)
+   
 3.1
 Merged from 3.0:
  * Avoid MV race during node decommission (CASSANDRA-10674)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/feaed2a4/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 66f6b71..bf9c2b8 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -406,7 +406,7 @@ public abstract class UnfilteredDeserializer
 
 public boolean hasNext()
 {
-// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below.
+// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below or the atom might be 
shadowed.
 while (next == null)
 {
 if (atoms.hasNext())
@@ -419,7 +419,8 @@ public abstract class UnfilteredDeserializer
 else
 {
 LegacyLayout.LegacyAtom atom = atoms.next();
-next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (!tombstoneTracker.isShadowed(atom))
+next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
 }
 }
 else if (tombstoneTracker.hasOpenTombstones())
@@ -498,7 +499,7 @@ public abstract class UnfilteredDeserializer
 if (isDone)
 return false;
 
-while (next == null)
+if (next == null)
 {
 next = readAtom();
 if (next == null)
@@ -506,9 +507,6 @@ public abstract class UnfilteredDeserializer
 isDone = true;
 return false;
 }
-
-if (tombstoneTracker.isShadowed(next))
-next = null;
 }
 return true;
 }
@@ -576,6 +574,7 @@ public abstract class UnfilteredDeserializer
  */
 public boolean isShadowed(LegacyLayout.LegacyAtom atom)
 {
+assert !hasClosingMarkerBefore(atom);
 long timestamp = atom.isCell() ? atom.asCell().timestamp : 
atom.asRangeTombstone().deletionTime.markedForDeleteAt();
 
 if (partitionDeletion.deletes(timestamp))

http://git-wip-us.apache.org/repos/asf/cassandra/blob/feaed2a4/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 3ce7d2c..0deaf44 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -958,13 +958,16 @@ public class CompactionManager implements 
CompactionManagerMBean
  

[1/2] cassandra git commit: Fix missing row data after range tombstone in legacy data

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0.2 [created] f0d235f58


Fix missing row data after range tombstone in legacy data

patch by blambov; reviewed by slebresne for CASSANDRA-10822

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a5c731b5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a5c731b5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a5c731b5

Branch: refs/heads/cassandra-3.0.2
Commit: a5c731b5908c3ccb19596e4b79f92ff93093089f
Parents: cf56770
Author: Branimir Lambov 
Authored: Thu Dec 10 15:33:15 2015 +0200
Committer: T Jake Luciani 
Committed: Mon Dec 14 11:56:23 2015 -0500

--
 CHANGES.txt  |  6 ++
 .../org/apache/cassandra/db/UnfilteredDeserializer.java  | 11 +--
 .../cassandra/db/compaction/CompactionManager.java   |  5 -
 3 files changed, 15 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a5c731b5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b95aa76..8ea6e91 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+3.0.2
+ * Fix upgrade data loss due to range tombstone deleting more data than then 
should
+   (CASSANDRA-10822)
+
 3.0.1
  * Avoid MV race during node decommission (CASSANDRA-10674)
  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
@@ -25,6 +29,8 @@
  * Keep the file open in trySkipCache (CASSANDRA-10669)
  * Updated trigger example (CASSANDRA-10257)
 Merged from 2.2:
+ * Fix regression on split size in CqlInputFormat (CASSANDRA-10835)
+ * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
  * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
  * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
  * Show CQL help in cqlsh in web browser (CASSANDRA-7225)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a5c731b5/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 66f6b71..bf9c2b8 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -406,7 +406,7 @@ public abstract class UnfilteredDeserializer
 
 public boolean hasNext()
 {
-// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below.
+// Note that we loop on next == null because 
TombstoneTracker.openNew() could return null below or the atom might be 
shadowed.
 while (next == null)
 {
 if (atoms.hasNext())
@@ -419,7 +419,8 @@ public abstract class UnfilteredDeserializer
 else
 {
 LegacyLayout.LegacyAtom atom = atoms.next();
-next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (!tombstoneTracker.isShadowed(atom))
+next = isRow(atom) ? readRow(atom) : 
tombstoneTracker.openNew(atom.asRangeTombstone());
 }
 }
 else if (tombstoneTracker.hasOpenTombstones())
@@ -498,7 +499,7 @@ public abstract class UnfilteredDeserializer
 if (isDone)
 return false;
 
-while (next == null)
+if (next == null)
 {
 next = readAtom();
 if (next == null)
@@ -506,9 +507,6 @@ public abstract class UnfilteredDeserializer
 isDone = true;
 return false;
 }
-
-if (tombstoneTracker.isShadowed(next))
-next = null;
 }
 return true;
 }
@@ -576,6 +574,7 @@ public abstract class UnfilteredDeserializer
  */
 public boolean isShadowed(LegacyLayout.LegacyAtom atom)
 {
+assert !hasClosingMarkerBefore(atom);
 long timestamp = atom.isCell() ? atom.asCell().timestamp : 
atom.asRangeTombstone().deletionTime.markedForDeleteAt();
 
 if (partitionDeletion.deletes(timestamp))

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056294#comment-15056294
 ] 

Ariel Weisberg commented on CASSANDRA-10855:


I don't object to swapping CLHM with a new implementation. As a matter of 
principle we should measure what impact this has so we can understand the yield 
of the optimizations in tiny LFU. We shouldn't be making changes for 
performance without measuring, but also so we can inform the evolution of OHC 
which solves the GC pressure issue.

I see on heap caches, memtables, and any other source of high volume survivor 
copying and promotion as a dead end.

We changed the format of the key cache for 3.0 so that it would be suitable for 
memory mapping, but didn't get the change in to actually use a memory mapped 
access method. Robert Stupp did write a lot of the code, but it was still using 
OHC as the cache. I suspect that will get us the 80%, but someone has to pick 
that up. [~JoshuaMcKenzie] we should think about if we want to schedule if we 
want to boost read performance and tail latencies due to reduced heap pressure.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10856) CHANGES and NEWS updated incorrectly when "thrift by default" was deprecated

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10856:


 Summary: CHANGES and NEWS updated incorrectly when "thrift by 
default" was deprecated
 Key: CASSANDRA-10856
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10856
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey


Thrift was no longer started by default in CASSANDRA-9319. This change affects 
2.2+, but the CHANGES and NEWS document the change as affecting 3.0+:

https://github.com/apache/cassandra/commit/fa4a020ac922fcdc0f3c2bebe35777cfa2e223c1

I think this is incorrect; [~iamaleksey] can you confirm?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: bump versions for 3.0.2

[jake]

bump versions for 3.0.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d235f5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d235f5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d235f5

Branch: refs/heads/cassandra-3.0.2
Commit: f0d235f58e31f6cfcf76ce33d05997ad41852520
Parents: a5c731b
Author: T Jake Luciani 
Authored: Mon Dec 14 12:02:15 2015 -0500
Committer: T Jake Luciani 
Committed: Mon Dec 14 12:02:15 2015 -0500

--
 NEWS.txt | 9 +
 build.xml| 2 +-
 debian/changelog | 6 ++
 3 files changed, 16 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d235f5/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index d41384b..476c0c1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,15 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+3.0.2
+=
+
+General
+---
+   - This release contains a critical fix regarding data loss when upgrading
+ a table with row tombstones.  
https://issues.apache.org/jira/browse/CASSANDRA-10822
+
+
 3.0.1
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d235f5/build.xml
--
diff --git a/build.xml b/build.xml
index 0420652..788cad5 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d235f5/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index f1232ab..8add839 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.0.2) unstable; urgency=medium
+
+  * New release 
+
+ -- Jake Luciani   Mon, 14 Dec 2015 11:58:12 -0500
+
 cassandra (3.0.1) unstable; urgency=medium
 
   * New release

[jira] [Created] (CASSANDRA-10864) Dropped mutations high until cluster restart

[Jeff Ferland (JIRA)]

Jeff Ferland created CASSANDRA-10864:


 Summary: Dropped mutations high until cluster restart
 Key: CASSANDRA-10864
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10864
 Project: Cassandra
  Issue Type: Bug
Reporter: Jeff Ferland


Originally raised and investigated in 
https://www.mail-archive.com/user@cassandra.apache.org/msg44586.html

Cause is still unclear, but a rolling restart has on two occasions since been 
performed to cope with dropped mutations and timed-out reads.

Pattern is indicative of some kind of code quality issue possibly involving 
locking operations. Stack flame graphs do not show a clear difference between 
restarts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056758#comment-15056758
 ] 

Michael Shuler commented on CASSANDRA-10861:


bq. It seems that connections crash

Could you provide some details on this? How can someone reproduce this?

{noformat}
14-Dec-2015 13:24:38.379 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)
{noformat}

The short snippet of logs you attached appears that your application is 
creating a large number of connections and running out of file descriptors. 
Have you set up the server(s) sysctl settings to handle the number of 
connections your application requires?

What is the Cassandra server state and what do the Cassandra server logs show 
when this occurs? What does the driver debug logging show? Have you gotten help 
from the java driver devs or is there a java driver JIRA ticket to look at?

This ticket is a little vague on details.

> Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1
> -
>
> Key: CASSANDRA-10861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Ubuntu 14.04.3 LTS 64 bits, Java build 1.8.0_66-b17, 
> tomcat 8.0.23
>Reporter: Carlos Scheidecker
> Fix For: 3.0.1
>
> Attachments: error_log_tomcat.txt
>
>
> Same dev environment with same application on Tomcat 8.0.23. However the dev 
> nodes have been upgraded to 3.0.0 and later to 3.0.1. The Cassandra driver is 
> version 3.0.0-beta1.
> It seems that connections crash, do not get cleared and it leads to a memory 
> leak stack overflow condition.
> Attached is an error log file from tomcat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Carlos Scheidecker (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056787#comment-15056787
 ] 

Carlos Scheidecker commented on CASSANDRA-10861:


Michael,

Sure, sorry if I had tried to be brief.

I cannot reproduce this. It only occurs after 1 or 2 days. But it occurs. Once 
Tomcat is restarted it is all good. As the API has not changed only the server 
and driver, the only errors I have are on the application driver side.

There are no errors on Cassandra side that I could find either on system.log 
nor debug.log. This seems to be a driver error.

The only thing I had from the same time on Cassandra debug.log is as follows:

INFO  [IndexSummaryManager:1] 2015-12-14 11:49:18,904 
IndexSummaryManager.java:257 - Redistributing index summaries
DEBUG [ScheduledTasks:1] 2015-12-14 12:49:15,198 ColumnFamilyStore.java:829 - 
Enqueuing flush of size_estimates: 5545698 (1%) on-heap, 0 (0%) off-heap
DEBUG [MemtableFlushWriter:141] 2015-12-14 12:49:15,198 Memtable.java:363 - 
Writing Memtable-size_estimates@1082810684(514.892KiB serialized bytes, 52200 
ops, 1%/0% of on/off-heap limit)
DEBUG [MemtableFlushWriter:141] 2015-12-14 12:49:15,226 Memtable.java:396 - 
Completed flushing 
/var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/ma-486-big-Data.db
 (351.674KiB) for commitlog position ReplayPosition(segmentId=1449870554433, 
position=15986025)
DEBUG [ScheduledTasks:1] 2015-12-14 12:49:17,758 ColumnFamilyStore.java:829 - 
Enqueuing flush of sstable_activity: 9294 (0%) on-heap, 0 (0%) off-heap
DEBUG [MemtableFlushWriter:142] 2015-12-14 12:49:17,758 Memtable.java:363 - 
Writing Memtable-sstable_activity@1052341053(1.162KiB serialized bytes, 168 
ops, 0%/0% of on/off-heap limit)
DEBUG [MemtableFlushWriter:142] 2015-12-14 12:49:17,770 Memtable.java:396 - 
Completed flushing 
/var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/ma-484-big-Data.db
 (1.297KiB) for commitlog position ReplayPosition(segmentId=1449870554433, 
position=15986025)
DEBUG [CompactionExecutor:3412] 2015-12-14 12:49:17,814 CompactionTask.java:146 
- Compacting (c2e5d160-a29b-11e5-bbad-cd29e2a8cd4b) 
[/var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/ma-484-big-Data.db:level=0,
 
/var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/ma-483-big-Data.db:level=0,
 
/var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/ma-482-big-Data.db:level=0,
 
/var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/ma-481-big-Data.db:level=0,
 ]
DEBUG [CompactionExecutor:3412] 2015-12-14 12:49:17,902 CompactionTask.java:217 
- Compacted (c2e5d160-a29b-11e5-bbad-cd29e2a8cd4b) 4 sstables to 
[/var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/ma-485-big,]
 to level=0.  2,555 bytes to 603 (~23% of original) in 88ms = 0.006535MB/s.  0 
total partitions merged to 14.  Partition merge counts were {1:8, 4:14, }
INFO  [IndexSummaryManager:1] 2015-12-14 12:49:18,914 
IndexSummaryManager.java:257 - Redistributing index summaries
DEBUG [ScheduledTasks:1] 2015-12-14 13:49:15,200 ColumnFamilyStore.java:829 - 
Enqueuing flush of size_estimates: 5545698 (1%) on-heap, 0 (0%) off-heap
DEBUG [MemtableFlushWriter:143] 2015-12-14 13:49:15,200 Memtable.java:363 - 
Writing Memtable-size_estimates@647480674(514.892KiB serialized bytes, 52200 
ops, 1%/0% of on/off-heap limit)
DEBUG [MemtableFlushWriter:143] 2015-12-14 13:49:15,220 Memtable.java:396 - 
Completed flushing 
/var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/ma-487-big-Data.db
 (351.674KiB) for commitlog position ReplayPosition(segmentId=1449870554433, 
position=20068341)
DEBUG [ScheduledTasks:1] 2015-12-14 13:49:17,756 ColumnFamilyStore.java:829 - 
Enqueuing flush of compaction_history: 1147 (0%) on-heap, 0 (0%) off-heap
DEBUG [MemtableFlushWriter:144] 2015-12-14 13:49:17,756 Memtable.java:363 - 
Writing Memtable-compaction_history@953834276(0.226KiB serialized bytes, 1 ops, 
0%/0% of on/off-heap limit)
DEBUG [ScheduledTasks:1] 2015-12-14 13:49:17,760 ColumnFamilyStore.java:829 - 
Enqueuing flush of sstable_activity: 9902 (0%) on-heap, 0 (0%) off-heap
DEBUG [MemtableFlushWriter:143] 2015-12-14 13:49:17,760 Memtable.java:363 - 
Writing Memtable-sstable_activity@481319531(1.193KiB serialized bytes, 172 ops, 
0%/0% of on/off-heap limit)
DEBUG [MemtableFlushWriter:144] 2015-12-14 13:49:17,772 Memtable.java:396 - 
Completed flushing 
/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/ma-208-big-Data.db
 (0.122KiB) for commitlog position ReplayPosition(segmentId=1449870554433, 
position=20068341)
DEBUG [MemtableFlushWriter:143] 2015-12-14 13:49:17,772 Memtable.java:396 - 
Completed flushing 

[jira] [Commented] (CASSANDRA-10858) test_copy_to_with_more_failures_than_max_attempts is failing

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056647#comment-15056647
 ] 

Philip Thompson commented on CASSANDRA-10858:
-

It appears to be happening to 
{{cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_child_process_crashing}}
 on 2.2+ as well.

> test_copy_to_with_more_failures_than_max_attempts is failing
> 
>
> Key: CASSANDRA-10858
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10858
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing, Tools
>Reporter: Philip Thompson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> {{cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_more_failures_than_max_attempts}}
>  which was introduced for CASSANDRA-9306 is failing when run on clusters 
> without vnodes. To reproduce, simply run the test with the environment 
> variable DISABLE_VNODES=true. 
> Here is the entire debug output:
> {code}
> dtest: DEBUG: cluster ccm directory: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY
> dtest: DEBUG: removing ccm cluster test at: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY
> dtest: DEBUG: clearing ssl stores from 
> [/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY] directory
> dtest: DEBUG: cluster ccm directory: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-CnOgA8
> dtest: DEBUG: Running stress
> dtest: DEBUG: Exporting to csv file: 
> /var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/tmpZYqB01 with 
> {"failing_range": {"start": 0, "end": 500, "num_failures": 
> 5}} and 3 max attempts
> dtest: DEBUG:
> Starting copy of keyspace1.standard1 with columns ['key', 'C0', 'C1', 'C2', 
> 'C3', 'C4'].
> Processed 1 rows; Written: 10503.508303 rows/s
> Processed 2 rows; Written: 11860.954046 rows/s
> Processed 3 rows; Written: 13068.388704 rows/s
> Processed 4 rows; Written: 16941.628006 rows/s
> Processed 5 rows; Written: 17609.109488 rows/s
> Processed 6 rows; Written: 19475.156238 rows/s
> Processed 7 rows; Written: 19976.978154 rows/s
> Processed 8 rows; Written: 19992.329090 rows/s
> Processed 9 rows; Written: 20623.291907 rows/s
> Processed 10 rows; Written: 21644.815363 rows/s
> Processed 10 rows; Written: 10822.407682 rows/s
> 10 rows exported in 5.816 seconds.
> {code}
> I assume this is related to the failure injection code in cqlsh not handling 
> fewer token ranges.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10859:


 Summary: AssertionError in nodetool cfhistograms
 Key: CASSANDRA-10859
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
 Fix For: 3.0.x, 3.x


{{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI {{trunk}} 
jobs:

http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/

This regression started in this build on CassCI and has failed consistently 
since then:

http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/

I can't find any recent dtest changes to this method, so I believe it's a 
Cassandra bug. Here's the changeset for the first failing build:

http://cassci.datastax.com/job/trunk_dtest/806/changes

Maybe something in the changes to the scripts here introduced the bug:

https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28

[~jjordan] could you have a look at this please?

This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10700) 2.1 sstableloader will fail if there are collections in the schema tables

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056662#comment-15056662
 ] 

T Jake Luciani commented on CASSANDRA-10700:


[fix | https://github.com/tjake/cassandra/tree/externalclient]
[testall | 
http://cassci.datastax.com/view/Dev/view/tjake/job/tjake-externalclient-testall/]
[dtest|http://cassci.datastax.com/job/tjake-externalclient-dtest/]

> 2.1 sstableloader will fail if there are collections in the schema tables
> -
>
> Key: CASSANDRA-10700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10700
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Tyler Hobbs
>Assignee: T Jake Luciani
> Fix For: 2.1.x
>
>
> In {{BulkLoader.ExternalClient}}, we use the Thrift {{execute_cql3_query()}} 
> method to read the system schema tables.  Because it's a Thrift connection, 
> we use the v2 protocol format for serializing data.  However, when we later 
> read the results with {{CFMetadata.fromThriftCqlRow()}}, we use the v3 
> protocol format to deserialize the results.  If there are any collections in 
> the results, such as entries in {{dropped_columns}}, the following error will 
> occur:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: null
> at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45-internal]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543) 
> ~[cassandra-all-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124)
>  ~[cassandra-all-2.1.1
> 1.jar:2.1.11]
> at 
> org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101)
>  ~[cassandra-all-
> 2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30)
>  ~[cassandra-all-2
> .1.11.jar:2.1.11]
> at 
> org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50)
>  ~[cassandra-all-2.1.
> 11.jar:2.1.11]
> at 
> org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68) 
> ~[cassandra-all-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:287)
>  ~[cassandra-all-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1833)
>  ~[cassandra-all-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1126) 
> ~[cassandra-all-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:330)
>  ~[cassandra-all-2.1.11.jar:na]
> ... 7 common frames omitted
> {noformat}
> I believe this only affects 2.1 due to the re-working of 
> BulkLoader/sstableloader in 2.2 and 3.0, but I haven't confirmed that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10858) test_copy_to_with_more_failures_than_max_attempts is failing

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-10858:
---

 Summary: test_copy_to_with_more_failures_than_max_attempts is 
failing
 Key: CASSANDRA-10858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10858
 Project: Cassandra
  Issue Type: Sub-task
  Components: Testing, Tools
Reporter: Philip Thompson
 Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x


{{cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_more_failures_than_max_attempts}}
 which was introduced for CASSANDRA-9306 is failing when run on clusters 
without vnodes. To reproduce, simply run the test with the environment variable 
DISABLE_VNODES=true. 

Here is the entire debug output:
{code}
dtest: DEBUG: cluster ccm directory: 
/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY
dtest: DEBUG: removing ccm cluster test at: 
/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY
dtest: DEBUG: clearing ssl stores from 
[/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-iJQECY] directory
dtest: DEBUG: cluster ccm directory: 
/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/dtest-CnOgA8
dtest: DEBUG: Running stress
dtest: DEBUG: Exporting to csv file: 
/var/folders/v3/z4wf_34n1q506_xjdy49gb78gn/T/tmpZYqB01 with 
{"failing_range": {"start": 0, "end": 500, "num_failures": 5}} 
and 3 max attempts
dtest: DEBUG:
Starting copy of keyspace1.standard1 with columns ['key', 'C0', 'C1', 'C2', 
'C3', 'C4'].
Processed 1 rows; Written: 10503.508303 rows/s
Processed 2 rows; Written: 11860.954046 rows/s
Processed 3 rows; Written: 13068.388704 rows/s
Processed 4 rows; Written: 16941.628006 rows/s
Processed 5 rows; Written: 17609.109488 rows/s
Processed 6 rows; Written: 19475.156238 rows/s
Processed 7 rows; Written: 19976.978154 rows/s
Processed 8 rows; Written: 19992.329090 rows/s
Processed 9 rows; Written: 20623.291907 rows/s
Processed 10 rows; Written: 21644.815363 rows/s
Processed 10 rows; Written: 10822.407682 rows/s
10 rows exported in 5.816 seconds.
{code}

I assume this is related to the failure injection code in cqlsh not handling 
fewer token ranges.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[4/4] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[blerer]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9bfe6135
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9bfe6135
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9bfe6135

Branch: refs/heads/trunk
Commit: 9bfe613579d3febda4e0c448a012c0e78a4d277b
Parents: a808769 94e8d62
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:37:27 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:37:39 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bfe6135/CHANGES.txt
--
diff --cc CHANGES.txt
index 10b6226,c30d35d..0ca5277
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,12 +1,24 @@@
 -3.0.2
 -Merged from 2.2
 +3.2
 + * Group pending compactions based on table (CASSANDRA-10718)
 + * Add compressor name in sstablemetadata output (CASSANDRA-9879)
 + * Fix type casting for counter columns (CASSANDRA-10824)
 + * Prevent running Cassandra as root (CASSANDRA-8142)
 + * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +Merged from 2.2:
   * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 - * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 - * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 - * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 - * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
  Merged from 2.1:
+  * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
  

[3/4] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/94e8d62c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/94e8d62c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/94e8d62c

Branch: refs/heads/trunk
Commit: 94e8d62cf8cea8918b01b038746be4b4e03af515
Parents: a8b9e53 263763a
Author: Benjamin Lerer 
Authored: Mon Dec 14 10:29:07 2015 +0100
Committer: Benjamin Lerer 
Committed: Mon Dec 14 10:29:17 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/stress/generate/values/Generator.java|  4 ++--
 .../apache/cassandra/stress/generate/values/Lists.java | 13 +++--
 .../apache/cassandra/stress/generate/values/Sets.java  | 10 +-
 4 files changed, 15 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/94e8d62c/CHANGES.txt
--
diff --cc CHANGES.txt
index e20f5ed,592ba0a..c30d35d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -6,40 -5,12 +6,41 @@@ Merged from 2.
   * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
   * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
  Merged from 2.1:
+  * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
 - * Fix Stress profile parsing on Windows (CASSANDRA-10808)
  
 -2.2.4
 +3.0.1
 + * Avoid MV race during node decommission (CASSANDRA-10674)
 + * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 + * Handle single-column deletions correction in materialized views
 +   when the column is part of the view primary key (CASSANDRA-10796)
 + * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
 + * Fix bug with range tombstones on reverse queries and test coverage for
 +   AbstractBTreePartition (CASSANDRA-10059)
 + * Remove 64k limit on collection elements (CASSANDRA-10374)
 + * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
 + * Fix NPE on stream read error (CASSANDRA-10771)
 + * Normalize cqlsh DESC output (CASSANDRA-10431)
 + * Rejects partition range deletions when columns are specified 
(CASSANDRA-10739)
 + * Fix error when saving cached key for old format sstable (CASSANDRA-10778)
 + * Invalidate prepared statements on DROP INDEX (CASSANDRA-10758)
 + * Fix SELECT statement with IN restrictions on partition key,
 +   ORDER BY and LIMIT (CASSANDRA-10729)
 + * Improve stress performance over 1k threads (CASSANDRA-7217)
 + * Wait for migration responses to complete before bootstrapping 
(CASSANDRA-10731)
 + * Unable to create a function with argument of type Inet (CASSANDRA-10741)
 + * Fix backward incompatibiliy in CqlInputFormat (CASSANDRA-10717)
 + * Correctly preserve deletion info on updated rows when notifying indexers
 +   of single-row deletions (CASSANDRA-10694)
 + * Notify indexers of partition delete during cleanup (CASSANDRA-10685)
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
 + * Fix regression on split size in CqlInputFormat (CASSANDRA-10835)
 + * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 + * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
 + * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
   * Show CQL help in cqlsh in web browser (CASSANDRA-7225)
   * Serialize on disk the proper SSTable compression ratio (CASSANDRA-10775)
   * Reject index queries while the index is building (CASSANDRA-8505)

[jira] [Commented] (CASSANDRA-9302) Optimize cqlsh COPY FROM, part 3

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056669#comment-15056669
 ] 

Adam Holmberg commented on CASSANDRA-9302:
--

Not a huge concern, but I noticed the {{INGESTRATE}} doesn't result in a very 
accurate real rate:
{code}
cassandra@cqlsh> COPY test.t from '2.1.csv' WITH HEADER = false AND 
REPORTFREQUENCY = '1' and CHUNKSIZE = '1' and INGESTRATE = '1';
...
Processed 7000 rows; Written: 4759.046292 rows/ss
7000 rows imported in 5.543 seconds.
cassandra@cqlsh> COPY test.t from '2.1.csv' WITH HEADER = false AND 
REPORTFREQUENCY = '1' and CHUNKSIZE = '1' and INGESTRATE = '100';
...
Processed 7000 rows; Written: 5297.179821 rows/ss
7000 rows imported in 3.972 seconds.
{code}

Am I misunderstanding its use?

Another minor observation: We sometimes get a double 's' in the rate message if 
the reported rate changes significant digits during the copy. We may want to 
use a fixed precision format for that.

> Optimize cqlsh COPY FROM, part 3
> 
>
> Key: CASSANDRA-9302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9302
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> We've had some discussion moving to Spark CSV import for bulk load in 3.x, 
> but people need a good bulk load tool now.  One option is to add a separate 
> Java bulk load tool (CASSANDRA-9048), but if we can match that performance 
> from cqlsh I would prefer to leave COPY FROM as the preferred option to which 
> we point people, rather than adding more tools that need to be supported 
> indefinitely.
> Previous work on COPY FROM optimization was done in CASSANDRA-7405 and 
> CASSANDRA-8225.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056411#comment-15056411
 ] 

Ariel Weisberg commented on CASSANDRA-9318:
---

Quick note. 65k mutations pending in the mutation stage. 7 memtables pending 
flush. [I hooked memtables pending flush into the backpressure 
mechanism.|https://github.com/apache/cassandra/commit/494eabf48ab48f1e86c058c0b583166ab39dcc39]
 That absolutely wrecked performance as throughput dropped to 0 zero 
periodically, but throughput is infinitely higher than when the database hasn't 
OOMed.

Kicked off a few performance runs to demonstrate what happens when you do have 
backpressure and you try various large limits on in flight memtables/requests.

[9318 w/backpressure 64m 8g heap memtables 
count|http://cstar.datastax.com/tests/id/fa769eec-a283-11e5-bbc9-0256e416528f]
[9318 w/backpressure 1g 8g heap memtables 
count|http://cstar.datastax.com/tests/id/4c52dd6e-a286-11e5-bbc9-0256e416528f]
[9318 w/backpressure 2g 8g heap memtables 
count|http://cstar.datastax.com/tests/id/b3d5b470-a286-11e5-bbc9-0256e416528f]

I am setting the point where backpressure turns off to almost the same limit as 
to when it turns on. This is smooths out performance just enough for stress to 
not constantly emit huge numbers of errors as writes time out because the 
database stops serving requests for a long time waiting for a memtable to flush.

With pressure from memtables somewhat accounted for the remaining source of 
pressure that can bring down a node is remotely delivered mutations. I can 
throw those into the calculation and add a listener that blocks reads from 
other cluster nodes. It's a nasty thing to do, but maybe not that different 
from OOM.

I am going to hack together something to force a node to be slow so I can 
demonstrate overwhelming it with remotely delivered mutations first.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9302) Optimize cqlsh COPY FROM, part 3

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056376#comment-15056376
 ] 

Adam Holmberg commented on CASSANDRA-9302:
--

bq. DCAware policy fixed it, now we have only one session.
Now that we're not choosing session based on replica host, we might further 
simplify {{split_batches}} to just group by partition key (i.e., no need for 
{{get_replica}}). Alternatively, if you want to send to a specific host other 
than one that load balancing would choose, we would need to borrow a connection 
and send directly on that (I don't think that's worth doing).

I find it a little awkward that numeric option values require quoting:
{code}
cassandra@cqlsh> COPY test.t FROM 'f.csv' WITH HEADER = false AND 
REPORTFREQUENCY = 100;
Improper COPY command.
cassandra@cqlsh> COPY test.t FROM 'f.csv' WITH HEADER = false AND 
REPORTFREQUENCY = '100';

Starting copy of test.t...
{code}
Is that a hard thing to change?

> Optimize cqlsh COPY FROM, part 3
> 
>
> Key: CASSANDRA-9302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9302
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> We've had some discussion moving to Spark CSV import for bulk load in 3.x, 
> but people need a good bulk load tool now.  One option is to add a separate 
> Java bulk load tool (CASSANDRA-9048), but if we can match that performance 
> from cqlsh I would prefer to leave COPY FROM as the preferred option to which 
> we point people, rather than adding more tools that need to be supported 
> indefinitely.
> Previous work on COPY FROM optimization was done in CASSANDRA-7405 and 
> CASSANDRA-8225.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Make tablehistograms accept the same syntax as tablestats

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 6bc363726 -> e5db3f2b1


Make tablehistograms accept the same syntax as tablestats

patch by Sunkret Hasdemir; reviewed by yukim for CASSANDRA-10149


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5db3f2b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5db3f2b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5db3f2b

Branch: refs/heads/trunk
Commit: e5db3f2b1d77a00308c0e2ef3b5460a8e636388c
Parents: 6bc3637
Author: Sukret Hasdemir 
Authored: Sun Aug 30 06:56:44 2015 -0400
Committer: Yuki Morishita 
Committed: Mon Dec 14 12:10:05 2015 -0600

--
 CHANGES.txt |  1 +
 .../tools/nodetool/TableHistograms.java | 23 +++-
 2 files changed, 19 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5db3f2b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d0ca2e..eadd387 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.2
+ * Make tablehistograms accept the same syntax as tablestats (CASSANDRA-10149)
  * Group pending compactions based on table (CASSANDRA-10718)
  * Add compressor name in sstablemetadata output (CASSANDRA-9879)
  * Fix type casting for counter columns (CASSANDRA-10824)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5db3f2b/src/java/org/apache/cassandra/tools/nodetool/TableHistograms.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/TableHistograms.java 
b/src/java/org/apache/cassandra/tools/nodetool/TableHistograms.java
index 23f1c43..f3f9b8a 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/TableHistograms.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/TableHistograms.java
@@ -34,16 +34,29 @@ import org.apache.commons.lang3.ArrayUtils;
 @Command(name = "tablehistograms", description = "Print statistic histograms 
for a given table")
 public class TableHistograms extends NodeToolCmd
 {
-@Arguments(usage = " ", description = "The keyspace and 
table name")
+@Arguments(usage = "  | ", description = 
"The keyspace and table name")
 private List args = new ArrayList<>();
 
 @Override
 public void execute(NodeProbe probe)
 {
-checkArgument(args.size() == 2, "tablehistograms requires keyspace and 
table name arguments");
-
-String keyspace = args.get(0);
-String table = args.get(1);
+String keyspace = null, table = null;
+if (args.size() == 2)
+{
+keyspace = args.get(0);
+table = args.get(1);
+}
+else if (args.size() == 1)
+{
+String[] input = args.get(0).split("\\.");
+checkArgument(input.length == 2, "tablehistograms requires 
keyspace and table name arguments");
+keyspace = input[0];
+table = input[1];
+}
+else
+{
+checkArgument(false, "tablehistograms requires keyspace and table 
name arguments");
+}
 
 // calculate percentile of row size and column count
 long[] estimatedPartitionSize = (long[]) 
probe.getColumnFamilyMetric(keyspace, table, "EstimatedPartitionSizeHistogram");

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Ben Manes (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056445#comment-15056445
 ] 

Ben Manes commented on CASSANDRA-10855:
---

A little background from private discussions with Jonathan and Robert leading 
to this ticket. If the performance analysis is positive then it provides a 
strong motivation to integrate W-TinyLFU into OHC. As Robert described it, the 
on-heap caches are low hanging fruit where the key cache is performance 
critical.

These caching projects have always been on my personal time as a research 
hobby. My original interest was on concurrency, since all other caches either 
use coarse locking or make sacrifices for throughput (e.g. lower hit rates, 
O(n) eviction). More recently I worked on improving hit rates, as described in 
that paper. I've tended to keep a narrow focus until solving a particularly 
hard problem and off-heap adds more complexity so it wasn't tackled. I'm a 
little disappointed that OHC didn't borrow ideas from CLHM, as the fundamentals 
are transferable, but perhaps we'll combine them all together in a future 
project.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056411#comment-15056411
 ] 

Ariel Weisberg edited comment on CASSANDRA-9318 at 12/14/15 6:20 PM:
-

Quick note. 65k mutations pending in the mutation stage. 7 memtables pending 
flush. [I hooked memtables pending flush into the backpressure 
mechanism.|https://github.com/apache/cassandra/commit/494eabf48ab48f1e86c058c0b583166ab39dcc39]
 That absolutely wrecked performance as throughput dropped to 0 zero 
periodically, but throughput is infinitely higher than when the database has 
OOMed.

Kicked off a few performance runs to demonstrate what happens when you do have 
backpressure and you try various large limits on in flight memtables/requests.

[9318 w/backpressure 64m 8g heap memtables 
count|http://cstar.datastax.com/tests/id/fa769eec-a283-11e5-bbc9-0256e416528f]
[9318 w/backpressure 1g 8g heap memtables 
count|http://cstar.datastax.com/tests/id/4c52dd6e-a286-11e5-bbc9-0256e416528f]
[9318 w/backpressure 2g 8g heap memtables 
count|http://cstar.datastax.com/tests/id/b3d5b470-a286-11e5-bbc9-0256e416528f]

I am setting the point where backpressure turns off to almost the same limit as 
to when it turns on. This is smooths out performance just enough for stress to 
not constantly emit huge numbers of errors as writes time out because the 
database stops serving requests for a long time waiting for a memtable to flush.

With pressure from memtables somewhat accounted for the remaining source of 
pressure that can bring down a node is remotely delivered mutations. I can 
throw those into the calculation and add a listener that blocks reads from 
other cluster nodes. It's a nasty thing to do, but maybe not that different 
from OOM.

I am going to hack together something to force a node to be slow so I can 
demonstrate overwhelming it with remotely delivered mutations first.


was (Author: aweisberg):
Quick note. 65k mutations pending in the mutation stage. 7 memtables pending 
flush. [I hooked memtables pending flush into the backpressure 
mechanism.|https://github.com/apache/cassandra/commit/494eabf48ab48f1e86c058c0b583166ab39dcc39]
 That absolutely wrecked performance as throughput dropped to 0 zero 
periodically, but throughput is infinitely higher than when the database hasn't 
OOMed.

Kicked off a few performance runs to demonstrate what happens when you do have 
backpressure and you try various large limits on in flight memtables/requests.

[9318 w/backpressure 64m 8g heap memtables 
count|http://cstar.datastax.com/tests/id/fa769eec-a283-11e5-bbc9-0256e416528f]
[9318 w/backpressure 1g 8g heap memtables 
count|http://cstar.datastax.com/tests/id/4c52dd6e-a286-11e5-bbc9-0256e416528f]
[9318 w/backpressure 2g 8g heap memtables 
count|http://cstar.datastax.com/tests/id/b3d5b470-a286-11e5-bbc9-0256e416528f]

I am setting the point where backpressure turns off to almost the same limit as 
to when it turns on. This is smooths out performance just enough for stress to 
not constantly emit huge numbers of errors as writes time out because the 
database stops serving requests for a long time waiting for a memtable to flush.

With pressure from memtables somewhat accounted for the remaining source of 
pressure that can bring down a node is remotely delivered mutations. I can 
throw those into the calculation and add a listener that blocks reads from 
other cluster nodes. It's a nasty thing to do, but maybe not that different 
from OOM.

I am going to hack together something to force a node to be slow so I can 
demonstrate overwhelming it with remotely delivered mutations first.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10847) Log a message when refusing a major compaction due to incremental repairedAt status

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10847:
-
Priority: Trivial  (was: Major)

> Log a message when refusing a major compaction due to incremental repairedAt 
> status
> ---
>
> Key: CASSANDRA-10847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10847
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Brandon Williams
>Assignee: Marcus Eriksson
>Priority: Trivial
>
> When you do a major compaction, but have some repaired sstables and some that 
> are not, it will correctly not compact them together.  However, it can be 
> somewhat confusing the operator as to *why* they aren't compacting together.  
> It would be beneficial, specifically when doing a major, to log that we 
> aren't going to do a full major because of this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10847) Log a message when refusing a major compaction due to incremental repairedAt status

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10847:
-
Assignee: Marcus Eriksson

> Log a message when refusing a major compaction due to incremental repairedAt 
> status
> ---
>
> Key: CASSANDRA-10847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10847
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Brandon Williams
>Assignee: Marcus Eriksson
>
> When you do a major compaction, but have some repaired sstables and some that 
> are not, it will correctly not compact them together.  However, it can be 
> somewhat confusing the operator as to *why* they aren't compacting together.  
> It would be beneficial, specifically when doing a major, to log that we 
> aren't going to do a full major because of this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10865) commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10865:


 Summary: commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 
2.1
 Key: CASSANDRA-10865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10865
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
 Fix For: 2.1.x


This test is failing hard on 2.1:

http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/commitlog_test/TestCommitLog/test_bad_crc/history/

In spite of having a bad commitlog, the node successfully starts. Is this a C* 
bug, or just something that wasn't implemented in 2.1? It seems to succeed on 
2.2+.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Carlos Scheidecker (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056803#comment-15056803
 ] 

Carlos Scheidecker commented on CASSANDRA-10861:


As you can see, looks like that heartbeat does not work on all nodes 
(192.168.1.15, 192.168.1.16 and 192.168.1.17) at first just 192.168.1.15 and 
then works well next.

These are from Tomcat logs:

13:12:33.904 [cluster1-nio-worker-0] DEBUG com.datastax.driver.core.Connection 
- Connection[/192.168.1.15:9042-1, inFlight=0, closed=false] was inactive for 
30 seconds, sending heartbeat
13:12:33.905 [cluster1-nio-worker-0] DEBUG com.datastax.driver.core.Connection 
- Connection[/192.168.1.15:9042-1, inFlight=0, closed=false] heartbeat query 
succeeded
14-Dec-2015 13:12:33.941 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.041 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.242 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.242 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.292 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.393 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.593 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:686)
at java.lang.Thread.run(Thread.java:745)

14-Dec-2015 13:12:34.994 SEVERE [http-nio-8080-Acceptor-0] 
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at 

[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10580:
-
Attachment: 0001-Metrics.patch

Patch for exposing this data as JMX metrics

> On dropped mutations, more details should be logged.
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> CASSANDRA-10580-Head.patch, Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056814#comment-15056814
 ] 

Michael Shuler commented on CASSANDRA-10861:


Max open files 4096 4096 files

> Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1
> -
>
> Key: CASSANDRA-10861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Ubuntu 14.04.3 LTS 64 bits, Java build 1.8.0_66-b17, 
> tomcat 8.0.23
>Reporter: Carlos Scheidecker
> Fix For: 3.0.1
>
> Attachments: error_log_tomcat.txt
>
>
> Same dev environment with same application on Tomcat 8.0.23. However the dev 
> nodes have been upgraded to 3.0.0 and later to 3.0.1. The Cassandra driver is 
> version 3.0.0-beta1.
> It seems that connections crash, do not get cleared and it leads to a memory 
> leak stack overflow condition.
> Attached is an error log file from tomcat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10580) On dropped mutations, more details should be logged.

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056808#comment-15056808
 ] 

Anubhav Kale edited comment on CASSANDRA-10580 at 12/14/15 10:12 PM:
-

Thanks for the pointers. Please take a look at the latest patch. I have tested 
it via Visual VM and the new metrics work well. 

It is interesting that I could not find any documentation that Metrics return 
results (.getSnapshot.GetMean()) in nanoseconds therefore callers must convert 
themselves. Noting this here so other Devs can save some time on this.

I'll file a separate JIRA for doing the same change on a per ks/cf basis. 

Let me know this makes sense, or more changes are needed. 


was (Author: anubhavk):
Patch for exposing this data as JMX metrics

> On dropped mutations, more details should be logged.
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> CASSANDRA-10580-Head.patch, Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10865) commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey reassigned CASSANDRA-10865:


Assignee: Jim Witschey

> commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1
> 
>
> Key: CASSANDRA-10865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Jim Witschey
> Fix For: 2.1.x
>
>
> This test is failing hard on 2.1:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/commitlog_test/TestCommitLog/test_bad_crc/history/
> In spite of having a bad commitlog, the node successfully starts. Is this a 
> C* bug, or just something that wasn't implemented in 2.1? It seems to succeed 
> on 2.2+.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10866) Column Family should expose latency metrics for dropped mutations.

[Anubhav Kale (JIRA)]

Anubhav Kale created CASSANDRA-10866:


 Summary: Column Family should expose latency metrics for dropped 
mutations.
 Key: CASSANDRA-10866
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10866
 Project: Cassandra
  Issue Type: Improvement
 Environment: PROD
Reporter: Anubhav Kale
Priority: Minor


Please take a look at the discussion in CASSANDRA-10580. This is opened so that 
the latency on dropped mutations is exposed as a metric on column families.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10865) commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056812#comment-15056812
 ] 

Joel Knighton commented on CASSANDRA-10865:
---

This test was added as part of [CASSANDRA-9749], which only went in to 2.2 and 
above.

It seems we shouldn't be running this on 2.1.

> commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1
> 
>
> Key: CASSANDRA-10865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
> Fix For: 2.1.x
>
>
> This test is failing hard on 2.1:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/commitlog_test/TestCommitLog/test_bad_crc/history/
> In spite of having a bad commitlog, the node successfully starts. Is this a 
> C* bug, or just something that wasn't implemented in 2.1? It seems to succeed 
> on 2.2+.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10865) commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056825#comment-15056825
 ] 

Jim Witschey commented on CASSANDRA-10865:
--

I've also reproduced this locally on Linux.

> commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1
> 
>
> Key: CASSANDRA-10865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
> Fix For: 2.1.x
>
>
> This test is failing hard on 2.1:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/commitlog_test/TestCommitLog/test_bad_crc/history/
> In spite of having a bad commitlog, the node successfully starts. Is this a 
> C* bug, or just something that wasn't implemented in 2.1? It seems to succeed 
> on 2.2+.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10865) commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056838#comment-15056838
 ] 

Jim Witschey commented on CASSANDRA-10865:
--

Brilliant, thanks [~jkni]. Assigning myself.

> commitlog_test.py:TestCommitLog.test_bad_crc fails on C* 2.1
> 
>
> Key: CASSANDRA-10865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
> Fix For: 2.1.x
>
>
> This test is failing hard on 2.1:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/commitlog_test/TestCommitLog/test_bad_crc/history/
> In spite of having a bad commitlog, the node successfully starts. Is this a 
> C* bug, or just something that wasn't implemented in 2.1? It seems to succeed 
> on 2.2+.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) On dropped mutations, more details should be logged.

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056843#comment-15056843
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

CASSANDRA-10866 is opened to track per CF work.

> On dropped mutations, more details should be logged.
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> CASSANDRA-10580-Head.patch, Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10867) thrift_tests.py:TestCQLAccesses.test_range_tombstone_and_static failing on C* 2.1

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10867:


 Summary: 
thrift_tests.py:TestCQLAccesses.test_range_tombstone_and_static failing on C* 
2.1
 Key: CASSANDRA-10867
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10867
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey


http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/thrift_tests/TestCQLAccesses/test_range_tombstone_and_static/history/

I haven't had enough experience with thrift or the thrift tests to debug this. 
It passes on 2.2+. I've reproduced this failure locally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10860) repair anticompaction tests are failing on 2.2+

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10860:

Description: 
{{repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test}} and 
{{repair_test.TestRepair.no_anticompaction_after_dclocal_repair_test}} are both 
failing on 2.2 and 3.0, with run on clusters not using vnodes. You can run the 
tests by setting DISABLE_VNODES=true.

These tests were added for CASSANDRA-10422.

When I run these locally, it appears from the logs that the repair is not 
running on every node, despite being specified in the nodetool command. See 
http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/168/testReport/ for 
an example failure. I can reproduce this consistently locally.

  was:
{{repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test}} and 
{{repair_test.TestRepair.no_anticompaction_after_dclocal_repair_test}} are both 
failing on 2.2 and 3.0, with run on clusters not using vnodes. You can run the 
tests by setting DISABLE_VNODES=true.

These tests were added for CASSANDRA-10422.

When I run these locally, it appears from the logs that the repair is not 
running on every node, despite being specified in the nodetool command.


> repair anticompaction tests are failing on 2.2+
> ---
>
> Key: CASSANDRA-10860
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10860
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Compaction, Testing
>Reporter: Philip Thompson
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> {{repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test}} 
> and {{repair_test.TestRepair.no_anticompaction_after_dclocal_repair_test}} 
> are both failing on 2.2 and 3.0, with run on clusters not using vnodes. You 
> can run the tests by setting DISABLE_VNODES=true.
> These tests were added for CASSANDRA-10422.
> When I run these locally, it appears from the logs that the repair is not 
> running on every node, despite being specified in the nodetool command. See 
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/168/testReport/ 
> for an example failure. I can reproduce this consistently locally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10863) HSHA test_closing_connections test still flaps on 3.0

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10863:


 Summary: HSHA test_closing_connections test still flaps on 3.0
 Key: CASSANDRA-10863
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10863
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
 Fix For: 3.0.x


The problem reported in CASSANDRA-10570 still seems to be happening on CassCI, 
as recently as a couple days ago:

http://cassci.datastax.com/job/cassandra-3.0_dtest/433/

[~carlyeks] The fix in #10570 was to increase how long we'd sleep in the tests. 
Should we just bump it up further, or does this make you suspect a larger 
problem here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Jeff Ferland (JIRA)]

Jeff Ferland created CASSANDRA-10862:


 Summary: LCS repair: compact tables before making available in L0
 Key: CASSANDRA-10862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
 Project: Cassandra
  Issue Type: Improvement
  Components: Compaction, Streaming and Messaging
Reporter: Jeff Ferland


When doing repair on a system with lots of mismatched ranges, the number of 
tables in L0 goes up dramatically, as correspondingly goes the number of tables 
referenced for a query. Latency increases dramatically in tandem.

Eventually all the copied tables are compacted down in L0, then copied into L1 
(which may be a very large copy), finally reducing the number of SSTables per 
query into the manageable range.

It seems to me that the cleanest answer is to compact after streaming, then 
mark tables available rather than marking available when the file itself is 
complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-10859:
---
   Assignee: Yuki Morishita
Component/s: Tools

CASSANDRA-10679 only changed scripts (how tools load config file etc) so I 
doubt it is the cause.
I'll take a look.

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x, 3.x
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10860) repair anticompaction tests are failing on 2.2+

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-10860:
---

 Summary: repair anticompaction tests are failing on 2.2+
 Key: CASSANDRA-10860
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10860
 Project: Cassandra
  Issue Type: Sub-task
  Components: Compaction, Testing
Reporter: Philip Thompson
 Fix For: 2.2.x, 3.0.x, 3.x


{{repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test}} and 
{{repair_test.TestRepair.no_anticompaction_after_dclocal_repair_test}} are both 
failing on 2.2 and 3.0, with run on clusters not using vnodes. You can run the 
tests by setting DISABLE_VNODES=true.

These tests were added for CASSANDRA-10422.

When I run these locally, it appears from the logs that the repair is not 
running on every node, despite being specified in the nodetool command.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10745) Deprecate PropertyFileSnitch

[Sean Durity (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056714#comment-15056714
 ] 

Sean Durity commented on CASSANDRA-10745:
-

I would not want to use GPFS. PropertyFileSnitch is easier to package and 
distribute because I can include the exact same file for every host. GPFS 
requires that files are (potentially) different on every host. Also, if I am on 
any particular host, I can view the full topology in the topology file. So, not 
a use case argument, exactly, but I would have to write logic to create the 
GPFS file on each node during deployment. Guess what I would include? Something 
very like the topology file.

> Deprecate PropertyFileSnitch
> 
>
> Key: CASSANDRA-10745
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10745
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Distributed Metadata
>Reporter: Paulo Motta
>Priority: Minor
>
> Opening this ticket to discuss deprecating PropertyFileSnitch, since it's 
> error-prone and more snitch code to maintain (See CASSANDRA-10243). Migration 
> from existing cluster with PropertyFileSnitch to GossipingPropertyFileSnitch 
> is straightforward.
> Is there any useful use case that can be achieved only with 
> PropertyFileSnitch?
> If not objections, we would add deprecation warnings in 2.2.x, 3.0.x, 3.2 and 
> deprecate in 3.4 or 3.6.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Carlos Scheidecker (JIRA)]

Carlos Scheidecker created CASSANDRA-10861:
--

 Summary: Memory leak with Cassadra java driver 3.0.0-beta1 and 
Cassandra 3.0.1
 Key: CASSANDRA-10861
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10861
 Project: Cassandra
  Issue Type: Bug
  Components: Streaming and Messaging
 Environment: Ubuntu 14.04.3 LTS 64 bits, Java build 1.8.0_66-b17, 
tomcat 8.0.23
Reporter: Carlos Scheidecker
 Fix For: 3.0.1
 Attachments: error_log_tomcat.txt

Same dev environment with same application on Tomcat 8.0.23. However the dev 
nodes have been upgraded to 3.0.0 and later to 3.0.1. The Cassandra driver is 
version 3.0.0-beta1.

It seems that connections crash, do not get cleared and it leads to a memory 
leak stack overflow condition.

Attached is an error log file from tomcat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10868) Skip supercolumns upgrade tests on jdk8

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10868:


 Summary: Skip supercolumns upgrade tests on jdk8
 Key: CASSANDRA-10868
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10868
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
Assignee: Jim Witschey


The tests in the {{upgrade_supercolumns_test}} dtest module fail when we test 
on JDK8 because they attempt to upgrade from 2.0, which will not compile on 
JDK8:

http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/160/testReport/upgrade_supercolumns_test/

[~rhatch] As we look at how we want to run upgrade tests in the future, we 
should consider this. In the meantime, I think the best way to deal with this 
might be to add something to the exclude files in {{conf/}}. That sound 
reasonable, or is there a better way to do this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Carlos Scheidecker (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056883#comment-15056883
 ] 

Carlos Scheidecker commented on CASSANDRA-10861:


Michael,

I had it increased on Tomcat updstart script to 5 let's see how it goes now.

Limit Soft Limit   Hard Limit   Units 
Max cpu time  unlimitedunlimitedseconds   
Max file size unlimitedunlimitedbytes 
Max data size unlimitedunlimitedbytes 
Max stack size8388608  unlimitedbytes 
Max core file size0unlimitedbytes 
Max resident set  unlimitedunlimitedbytes 
Max processes 256854   256854   processes 
Max open files55files 
Max locked memory 6553665536bytes 
Max address space unlimitedunlimitedbytes 
Max file locksunlimitedunlimitedlocks 
Max pending signals   256854   256854   signals   
Max msgqueue size 819200   819200   bytes 
Max nice priority 00
Max realtime priority 00
Max realtime timeout  unlimitedunlimitedus

> Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1
> -
>
> Key: CASSANDRA-10861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Ubuntu 14.04.3 LTS 64 bits, Java build 1.8.0_66-b17, 
> tomcat 8.0.23
>Reporter: Carlos Scheidecker
> Fix For: 3.0.1
>
> Attachments: error_log_tomcat.txt
>
>
> Same dev environment with same application on Tomcat 8.0.23. However the dev 
> nodes have been upgraded to 3.0.0 and later to 3.0.1. The Cassandra driver is 
> version 3.0.0-beta1.
> It seems that connections crash, do not get cleared and it leads to a memory 
> leak stack overflow condition.
> Attached is an error log file from tomcat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10861) Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-10861.

   Resolution: Not A Problem
Fix Version/s: (was: 3.0.1)

> Memory leak with Cassadra java driver 3.0.0-beta1 and Cassandra 3.0.1
> -
>
> Key: CASSANDRA-10861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Ubuntu 14.04.3 LTS 64 bits, Java build 1.8.0_66-b17, 
> tomcat 8.0.23
>Reporter: Carlos Scheidecker
> Attachments: error_log_tomcat.txt
>
>
> Same dev environment with same application on Tomcat 8.0.23. However the dev 
> nodes have been upgraded to 3.0.0 and later to 3.0.1. The Cassandra driver is 
> version 3.0.0-beta1.
> It seems that connections crash, do not get cleared and it leads to a memory 
> leak stack overflow condition.
> Attached is an error log file from tomcat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10869) paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions dtest fails on 2.1

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10869:


 Summary: 
paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions dtest 
fails on 2.1
 Key: CASSANDRA-10869
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10869
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey


This test is failing hard on 2.1. Here is its history on the JDK8 job for 
cassandra-2.1:

http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/lastCompletedBuild/testReport/paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/

and on the JDK7 job:

http://cassci.datastax.com/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/

It fails because a read times out after ~1.5 minutes. If this is a test error, 
it's specific to 2.1, because the test passes consistently on newer versions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10869) paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions dtest fails on 2.1

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056909#comment-15056909
 ] 

Jim Witschey commented on CASSANDRA-10869:
--

I've reproduced this locally on Linux, so this isn't just an environmental 
issue.

> paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions dtest 
> fails on 2.1
> --
>
> Key: CASSANDRA-10869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10869
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>
> This test is failing hard on 2.1. Here is its history on the JDK8 job for 
> cassandra-2.1:
> http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/lastCompletedBuild/testReport/paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> and on the JDK7 job:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> It fails because a read times out after ~1.5 minutes. If this is a test 
> error, it's specific to 2.1, because the test passes consistently on newer 
> versions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056940#comment-15056940
 ] 

Ariel Weisberg commented on CASSANDRA-9318:
---

Further review shows that with an 8 gigabyte heap backpressure isn't getting a 
chance to kick in. Several memtables end up being queued at once for flushing 
but the peak memory utilization only a few hundred megabytes. I am back to not 
having a workload that demonstrate that backpressure has value.

I tried artificially constraining disk throughput to 32 megabytes/sec and ended 
up hitting OOM in hints. Hints appear to allocate ByteBuffers indefinitely so 
they can't provide backpressure based on available disk capacity/throughput 
anymore. Hints are submitted directly from the contributing thread so there is 
no such thing as in-flight hint anymore. I'll clean up the existing hint 
overload protection so that it kicks in based on number of pending buffers and 
see where that gets me.

I am guessing that in practice hint overload is going to be a lot harder to hit 
now that it is based on flat files. Clearly sequential IO performance outstrips 
the ability to process even large incoming mutations.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10870) pushed_notifications_test.py:TestPushedNotifications.restart_node_test flapping on C* 2.1

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10870:


 Summary: 
pushed_notifications_test.py:TestPushedNotifications.restart_node_test flapping 
on C* 2.1
 Key: CASSANDRA-10870
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10870
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
 Fix For: 2.1.x


This test flaps on CassCI on 2.1. [~aboudreault] Do I remember correctly that 
you did some work on these tests in the past few months? If so, could you have 
a look and see if there's some assumption the test makes that don't hold for 
2.1?

Oddly, it fails frequently under JDK8:

http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/lastCompletedBuild/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test/history/

but less frequently on JDK7:

http://cassci.datastax.com/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test/history/




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)