[jira] [Commented] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978374#comment-14978374
 ] 

Stefania commented on CASSANDRA-10089:
--

[~benedict], thanks for looking into this.

You mean to say that the _happens-before_ edge is not guaranteed across keys, 
not even with CHM? 

The unit test accesses the map in the same way as 
{{Gossiper.getStateForVersionBiggerThan()}} does. At least that was the 
intention. It seems to reliably pass with CHM but maybe it is just luck. Is the 
only way around this problem a lock around the entire map or to give up on the 
map and pay O(N) for accessing states?

> NullPointerException in Gossip handleStateNormal
> 
>
> Key: CASSANDRA-10089
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10089
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.1
>
> Attachments: node1_debug.log, node2_debug.log, node3_debug.log
>
>
> Whilst comparing dtests for CASSANDRA-9970 I found [this failing 
> dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/]
>  in 2.2:
> {code}
> Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 
> 15:39:57,873 CassandraDaemon.java:183 - Exception in thread 
> Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat 
> org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>  ~[main/:na] \tat 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
> ~[main/:na] \tat 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_80] \tat 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]']
> {code}
> I wasn't able to find it on unpatched branches  but it is clearly not related 
> to CASSANDRA-9970, if anything it could have been a side effect of 
> CASSANDRA-9871.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[snazy]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/734a3bfa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/734a3bfa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/734a3bfa

Branch: refs/heads/trunk
Commit: 734a3bfa782bb0489878cd14c2f589d662c22f11
Parents: 5a356a7 32a6f20
Author: Robert Stupp 
Authored: Wed Oct 28 14:18:25 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:18:25 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  5 ++
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 77 ++--
 5 files changed, 78 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/CHANGES.txt
--
diff --cc CHANGES.txt
index 9021e9d,998dd22..c4f606f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 -2.1.12
 +2.2.4
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
   * Do STCS in DTCS windows (CASSANDRA-10276)
   * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/NEWS.txt
--
diff --cc NEWS.txt
index 028d26d,c6ea6c0..94567d6
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,23 -13,21 +13,28 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
  
 -2.1.12
 -==
 +2.2.4
 +=
 +
 +Deprecation
 +---
 +- Pig support has been deprecated, and will be removed in 3.0.
 +  Please see CASSANDRA-10542 for more details.
 +
 +Operations
 +--
 +- Switching racks is no longer an allowed operation on a node which has
 +  data. Instead, the node will need to be decommissioned and 
rebootstrapped.
 +  If moving from the SimpleSnitch, make sure the rack containing all 
current
 +  nodes is named "rack1".
  
+ New features
+ 
+ - Native protocol server now allows both SSL and non-SSL connections on
+   the same port.
 -- Switching racks is no longer an allowed operation on a node which has
 -  data. Instead, the node will need to be decommissioned and 
rebootstrapped.
 -  If moving from the SimpleSnitch, make sure the rack containing all 
current
 -  nodes is named "rack1". To override this behavior when manually wiping
 -  the node and bootstrapping, use -Dcassandra.ignore_rack=true.
+ 
  
 -2.1.11
 +2.2.3
  =
  
  Upgrading

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/conf/cassandra.yaml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/src/java/org/apache/cassandra/transport/Server.java
--
diff --cc src/java/org/apache/cassandra/transport/Server.java
index d610bff,02f17b0..c56564c
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@@ -30,20 -30,17 +30,22 @@@ import java.util.concurrent.atomic.Atom
  import javax.net.ssl.SSLContext;
  import javax.net.ssl.SSLEngine;
  
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import io.netty.bootstrap.ServerBootstrap;
+ import io.netty.buffer.ByteBuf;
 +import io.netty.channel.*;
  import io.netty.channel.epoll.Epoll;
  import io.netty.channel.epoll.EpollEventLoopGroup;
  import io.netty.channel.epoll.EpollServerSocketChannel;
 -import io.netty.handler.codec.ByteToMessageDecoder;
 -import io.netty.util.Version;
 -import org.slf4j.Logger;
 -import org.slf4j.LoggerFactory;
 -
 +import io.netty.channel.group.ChannelGroup;
 +import io.netty.channel.group.DefaultChannelGroup;
  import io.netty.channel.nio.NioEventLoopGroup;
  import io.netty.channel.socket.nio.NioServerSocketChannel;
++import io.netty.handler.codec.ByteToMessageDecoder;
 +import io.netty.handler.ssl.SslHandler;
 +import io.netty.util.Version;
  import io.netty.util.concurrent.EventExecutor;
  import 

cassandra git commit: Support encrypted and plain traffic on the same port

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 cedcf07c5 -> 32a6f2059


Support encrypted and plain traffic on the same port

patch by Norman Maurer; reviewed by Robert Stupp for CASSANDRA-10559


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32a6f205
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32a6f205
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32a6f205

Branch: refs/heads/cassandra-2.1
Commit: 32a6f2059aa3888fc0223cabcdff2ea2c9b97b21
Parents: cedcf07
Author: Norman Maurer 
Authored: Wed Oct 28 14:17:18 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:17:18 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  2 +
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 76 ++--
 5 files changed, 75 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5b46eac..998dd22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
  * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 67a545b..c6ea6c0 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,6 +18,8 @@ using the provided 'sstableupgrade' tool.
 
 New features
 
+- Native protocol server now allows both SSL and non-SSL connections on
+  the same port.
 - Switching racks is no longer an allowed operation on a node which has
   data. Instead, the node will need to be decommissioned and 
rebootstrapped.
   If moving from the SimpleSnitch, make sure the rack containing all 
current

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index e0ef878..0d0282b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -781,6 +781,8 @@ server_encryption_options:
 # enable or disable client/server encryption.
 client_encryption_options:
 enabled: false
+# If enabled and optional is set to true encrypted and unencrypted 
connections are handled.
+optional: false
 keystore: conf/.keystore
 keystore_password: cassandra
 # require_client_auth: false

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/config/EncryptionOptions.java
--
diff --git a/src/java/org/apache/cassandra/config/EncryptionOptions.java 
b/src/java/org/apache/cassandra/config/EncryptionOptions.java
index 945a15b..31f8b4a 100644
--- a/src/java/org/apache/cassandra/config/EncryptionOptions.java
+++ b/src/java/org/apache/cassandra/config/EncryptionOptions.java
@@ -36,6 +36,7 @@ public abstract class EncryptionOptions
 public static class ClientEncryptionOptions extends EncryptionOptions
 {
 public boolean enabled = false;
+public boolean optional = false;
 }
 
 public static class ServerEncryptionOptions extends EncryptionOptions

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index c21a669..02f17b0 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -22,6 +22,7 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.util.EnumMap;
+import java.util.List;
 import java.util.Map;
 import java.util.concurrent.Callable;
 import java.util.concurrent.ConcurrentHashMap;
@@ -29,9 +30,11 @@ import java.util.concurrent.atomic.AtomicBoolean;
 import javax.net.ssl.SSLContext;
 import javax.net.ssl.SSLEngine;
 
+import io.netty.buffer.ByteBuf;
 import io.netty.channel.epoll.Epoll;
 import io.netty.channel.epoll.EpollEventLoopGroup;
 import io.netty.channel.epoll.EpollServerSocketChannel;
+import 

[1/2] cassandra git commit: Support encrypted and plain traffic on the same port

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 5a356a706 -> 734a3bfa7


Support encrypted and plain traffic on the same port

patch by Norman Maurer; reviewed by Robert Stupp for CASSANDRA-10559


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32a6f205
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32a6f205
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32a6f205

Branch: refs/heads/cassandra-2.2
Commit: 32a6f2059aa3888fc0223cabcdff2ea2c9b97b21
Parents: cedcf07
Author: Norman Maurer 
Authored: Wed Oct 28 14:17:18 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:17:18 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  2 +
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 76 ++--
 5 files changed, 75 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5b46eac..998dd22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
  * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 67a545b..c6ea6c0 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,6 +18,8 @@ using the provided 'sstableupgrade' tool.
 
 New features
 
+- Native protocol server now allows both SSL and non-SSL connections on
+  the same port.
 - Switching racks is no longer an allowed operation on a node which has
   data. Instead, the node will need to be decommissioned and 
rebootstrapped.
   If moving from the SimpleSnitch, make sure the rack containing all 
current

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index e0ef878..0d0282b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -781,6 +781,8 @@ server_encryption_options:
 # enable or disable client/server encryption.
 client_encryption_options:
 enabled: false
+# If enabled and optional is set to true encrypted and unencrypted 
connections are handled.
+optional: false
 keystore: conf/.keystore
 keystore_password: cassandra
 # require_client_auth: false

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/config/EncryptionOptions.java
--
diff --git a/src/java/org/apache/cassandra/config/EncryptionOptions.java 
b/src/java/org/apache/cassandra/config/EncryptionOptions.java
index 945a15b..31f8b4a 100644
--- a/src/java/org/apache/cassandra/config/EncryptionOptions.java
+++ b/src/java/org/apache/cassandra/config/EncryptionOptions.java
@@ -36,6 +36,7 @@ public abstract class EncryptionOptions
 public static class ClientEncryptionOptions extends EncryptionOptions
 {
 public boolean enabled = false;
+public boolean optional = false;
 }
 
 public static class ServerEncryptionOptions extends EncryptionOptions

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index c21a669..02f17b0 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -22,6 +22,7 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.util.EnumMap;
+import java.util.List;
 import java.util.Map;
 import java.util.concurrent.Callable;
 import java.util.concurrent.ConcurrentHashMap;
@@ -29,9 +30,11 @@ import java.util.concurrent.atomic.AtomicBoolean;
 import javax.net.ssl.SSLContext;
 import javax.net.ssl.SSLEngine;
 
+import io.netty.buffer.ByteBuf;
 import io.netty.channel.epoll.Epoll;
 import io.netty.channel.epoll.EpollEventLoopGroup;
 import io.netty.channel.epoll.EpollServerSocketChannel;
+import 

[1/3] cassandra git commit: Support encrypted and plain traffic on the same port

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 13e9fd55f -> 690372755


Support encrypted and plain traffic on the same port

patch by Norman Maurer; reviewed by Robert Stupp for CASSANDRA-10559


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32a6f205
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32a6f205
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32a6f205

Branch: refs/heads/cassandra-3.0
Commit: 32a6f2059aa3888fc0223cabcdff2ea2c9b97b21
Parents: cedcf07
Author: Norman Maurer 
Authored: Wed Oct 28 14:17:18 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:17:18 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  2 +
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 76 ++--
 5 files changed, 75 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5b46eac..998dd22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
  * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 67a545b..c6ea6c0 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,6 +18,8 @@ using the provided 'sstableupgrade' tool.
 
 New features
 
+- Native protocol server now allows both SSL and non-SSL connections on
+  the same port.
 - Switching racks is no longer an allowed operation on a node which has
   data. Instead, the node will need to be decommissioned and 
rebootstrapped.
   If moving from the SimpleSnitch, make sure the rack containing all 
current

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index e0ef878..0d0282b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -781,6 +781,8 @@ server_encryption_options:
 # enable or disable client/server encryption.
 client_encryption_options:
 enabled: false
+# If enabled and optional is set to true encrypted and unencrypted 
connections are handled.
+optional: false
 keystore: conf/.keystore
 keystore_password: cassandra
 # require_client_auth: false

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/config/EncryptionOptions.java
--
diff --git a/src/java/org/apache/cassandra/config/EncryptionOptions.java 
b/src/java/org/apache/cassandra/config/EncryptionOptions.java
index 945a15b..31f8b4a 100644
--- a/src/java/org/apache/cassandra/config/EncryptionOptions.java
+++ b/src/java/org/apache/cassandra/config/EncryptionOptions.java
@@ -36,6 +36,7 @@ public abstract class EncryptionOptions
 public static class ClientEncryptionOptions extends EncryptionOptions
 {
 public boolean enabled = false;
+public boolean optional = false;
 }
 
 public static class ServerEncryptionOptions extends EncryptionOptions

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index c21a669..02f17b0 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -22,6 +22,7 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.util.EnumMap;
+import java.util.List;
 import java.util.Map;
 import java.util.concurrent.Callable;
 import java.util.concurrent.ConcurrentHashMap;
@@ -29,9 +30,11 @@ import java.util.concurrent.atomic.AtomicBoolean;
 import javax.net.ssl.SSLContext;
 import javax.net.ssl.SSLEngine;
 
+import io.netty.buffer.ByteBuf;
 import io.netty.channel.epoll.Epoll;
 import io.netty.channel.epoll.EpollEventLoopGroup;
 import io.netty.channel.epoll.EpollServerSocketChannel;
+import 

[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[snazy]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/69037275
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/69037275
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/69037275

Branch: refs/heads/cassandra-3.0
Commit: 69037275568d6fd753ee2ee40e7a5ac1751f459b
Parents: 13e9fd5 734a3bf
Author: Robert Stupp 
Authored: Wed Oct 28 14:20:02 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:20:02 2015 +0100

--
 CHANGES.txt | 2 +-
 NEWS.txt| 5 +
 2 files changed, 6 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/69037275/CHANGES.txt
--
diff --cc CHANGES.txt
index 76b1a55,c4f606f..b4cc43b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,13 +1,20 @@@
 -2.2.4
 +3.0
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
-  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
   * Do STCS in DTCS windows (CASSANDRA-10276)
 - * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)
   * Fix paging issues with partitions containing only static columns data 
(CASSANDRA-10381)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/69037275/NEWS.txt
--

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[snazy]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/734a3bfa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/734a3bfa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/734a3bfa

Branch: refs/heads/cassandra-2.2
Commit: 734a3bfa782bb0489878cd14c2f589d662c22f11
Parents: 5a356a7 32a6f20
Author: Robert Stupp 
Authored: Wed Oct 28 14:18:25 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:18:25 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  5 ++
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 77 ++--
 5 files changed, 78 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/CHANGES.txt
--
diff --cc CHANGES.txt
index 9021e9d,998dd22..c4f606f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 -2.1.12
 +2.2.4
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
   * Do STCS in DTCS windows (CASSANDRA-10276)
   * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/NEWS.txt
--
diff --cc NEWS.txt
index 028d26d,c6ea6c0..94567d6
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,23 -13,21 +13,28 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
  
 -2.1.12
 -==
 +2.2.4
 +=
 +
 +Deprecation
 +---
 +- Pig support has been deprecated, and will be removed in 3.0.
 +  Please see CASSANDRA-10542 for more details.
 +
 +Operations
 +--
 +- Switching racks is no longer an allowed operation on a node which has
 +  data. Instead, the node will need to be decommissioned and 
rebootstrapped.
 +  If moving from the SimpleSnitch, make sure the rack containing all 
current
 +  nodes is named "rack1".
  
+ New features
+ 
+ - Native protocol server now allows both SSL and non-SSL connections on
+   the same port.
 -- Switching racks is no longer an allowed operation on a node which has
 -  data. Instead, the node will need to be decommissioned and 
rebootstrapped.
 -  If moving from the SimpleSnitch, make sure the rack containing all 
current
 -  nodes is named "rack1". To override this behavior when manually wiping
 -  the node and bootstrapping, use -Dcassandra.ignore_rack=true.
+ 
  
 -2.1.11
 +2.2.3
  =
  
  Upgrading

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/conf/cassandra.yaml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/src/java/org/apache/cassandra/transport/Server.java
--
diff --cc src/java/org/apache/cassandra/transport/Server.java
index d610bff,02f17b0..c56564c
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@@ -30,20 -30,17 +30,22 @@@ import java.util.concurrent.atomic.Atom
  import javax.net.ssl.SSLContext;
  import javax.net.ssl.SSLEngine;
  
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import io.netty.bootstrap.ServerBootstrap;
+ import io.netty.buffer.ByteBuf;
 +import io.netty.channel.*;
  import io.netty.channel.epoll.Epoll;
  import io.netty.channel.epoll.EpollEventLoopGroup;
  import io.netty.channel.epoll.EpollServerSocketChannel;
 -import io.netty.handler.codec.ByteToMessageDecoder;
 -import io.netty.util.Version;
 -import org.slf4j.Logger;
 -import org.slf4j.LoggerFactory;
 -
 +import io.netty.channel.group.ChannelGroup;
 +import io.netty.channel.group.DefaultChannelGroup;
  import io.netty.channel.nio.NioEventLoopGroup;
  import io.netty.channel.socket.nio.NioServerSocketChannel;
++import io.netty.handler.codec.ByteToMessageDecoder;
 +import io.netty.handler.ssl.SslHandler;
 +import io.netty.util.Version;
  import io.netty.util.concurrent.EventExecutor;
  import 

[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[snazy]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/734a3bfa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/734a3bfa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/734a3bfa

Branch: refs/heads/cassandra-3.0
Commit: 734a3bfa782bb0489878cd14c2f589d662c22f11
Parents: 5a356a7 32a6f20
Author: Robert Stupp 
Authored: Wed Oct 28 14:18:25 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:18:25 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  5 ++
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 77 ++--
 5 files changed, 78 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/CHANGES.txt
--
diff --cc CHANGES.txt
index 9021e9d,998dd22..c4f606f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 -2.1.12
 +2.2.4
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
   * Do STCS in DTCS windows (CASSANDRA-10276)
   * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/NEWS.txt
--
diff --cc NEWS.txt
index 028d26d,c6ea6c0..94567d6
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,23 -13,21 +13,28 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
  
 -2.1.12
 -==
 +2.2.4
 +=
 +
 +Deprecation
 +---
 +- Pig support has been deprecated, and will be removed in 3.0.
 +  Please see CASSANDRA-10542 for more details.
 +
 +Operations
 +--
 +- Switching racks is no longer an allowed operation on a node which has
 +  data. Instead, the node will need to be decommissioned and 
rebootstrapped.
 +  If moving from the SimpleSnitch, make sure the rack containing all 
current
 +  nodes is named "rack1".
  
+ New features
+ 
+ - Native protocol server now allows both SSL and non-SSL connections on
+   the same port.
 -- Switching racks is no longer an allowed operation on a node which has
 -  data. Instead, the node will need to be decommissioned and 
rebootstrapped.
 -  If moving from the SimpleSnitch, make sure the rack containing all 
current
 -  nodes is named "rack1". To override this behavior when manually wiping
 -  the node and bootstrapping, use -Dcassandra.ignore_rack=true.
+ 
  
 -2.1.11
 +2.2.3
  =
  
  Upgrading

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/conf/cassandra.yaml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/734a3bfa/src/java/org/apache/cassandra/transport/Server.java
--
diff --cc src/java/org/apache/cassandra/transport/Server.java
index d610bff,02f17b0..c56564c
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@@ -30,20 -30,17 +30,22 @@@ import java.util.concurrent.atomic.Atom
  import javax.net.ssl.SSLContext;
  import javax.net.ssl.SSLEngine;
  
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import io.netty.bootstrap.ServerBootstrap;
+ import io.netty.buffer.ByteBuf;
 +import io.netty.channel.*;
  import io.netty.channel.epoll.Epoll;
  import io.netty.channel.epoll.EpollEventLoopGroup;
  import io.netty.channel.epoll.EpollServerSocketChannel;
 -import io.netty.handler.codec.ByteToMessageDecoder;
 -import io.netty.util.Version;
 -import org.slf4j.Logger;
 -import org.slf4j.LoggerFactory;
 -
 +import io.netty.channel.group.ChannelGroup;
 +import io.netty.channel.group.DefaultChannelGroup;
  import io.netty.channel.nio.NioEventLoopGroup;
  import io.netty.channel.socket.nio.NioServerSocketChannel;
++import io.netty.handler.codec.ByteToMessageDecoder;
 +import io.netty.handler.ssl.SslHandler;
 +import io.netty.util.Version;
  import io.netty.util.concurrent.EventExecutor;
  import 

[jira] [Updated] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock

[Mike Adamson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Adamson updated CASSANDRA-10608:
-
Priority: Minor  (was: Major)

> Adding a dynamic column to a compact storage table with the same name as the 
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
>
> The reproduction steps for this are as follows: 
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy', 
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column, 
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}} 
> containing a {{PartitionColumns}} containing the partition key 
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether 
> the {{ColumnDefinition}} is a partition key 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock

[Mike Adamson (JIRA)]

Mike Adamson created CASSANDRA-10608:


 Summary: Adding a dynamic column to a compact storage table with 
the same name as the partition key causes a memtable flush deadlock
 Key: CASSANDRA-10608
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Mike Adamson
 Fix For: 3.0.0


The reproduction steps for this are as follows: 
# Create the following schema:
{noformat}
CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy', 
'replication_factor': '1'};
CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
{noformat}
# Using the thrift client execute the following:
{noformat}
Column column = new Column(ByteBufferUtil.bytes("k"));
column.setValue(ByteBufferUtil.bytes(1));
column.setTimestamp(1);
client.insert(key, new ColumnParent(compactCf), column, 
ConsistencyLevel.ONE);
{noformat}

This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}} 
containing a {{PartitionColumns}} containing the partition key 
{{ColumnDefinition}}.

This happens because {{LegacyLayout.decodeCellName}} does not check whether the 
{{ColumnDefinition}} is a partition key 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-10559) Support encrypted and plain traffic on the same port

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp reopened CASSANDRA-10559:
--

Reopened, as we have a patch against 2.1 (+2.2), too.

> Support encrypted and plain traffic on the same port
> 
>
> Key: CASSANDRA-10559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10559
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Fix For: 3.0.0
>
> Attachments: 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-2.1.patch, 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-na.patch
>
>
> To be able to migrate clusters in a rolling way from plain to encrypted 
> traffic it would be very helpful if we could have Cassandra accept both on 
> the same port. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10559) Support encrypted and plain traffic on the same port

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978272#comment-14978272
 ] 

Robert Stupp commented on CASSANDRA-10559:
--

To be able to configure different ports for SSL and non-SSL clients might be an 
operational requirement.

Unfortunately MS still uses old, blocking I/O - there's CASSANDRA-8457 to 
change that.

> Support encrypted and plain traffic on the same port
> 
>
> Key: CASSANDRA-10559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10559
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Fix For: 3.0.0
>
> Attachments: 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-2.1.patch, 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-na.patch
>
>
> To be able to migrate clusters in a rolling way from plain to encrypted 
> traffic it would be very helpful if we could have Cassandra accept both on 
> the same port. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Jeff Griffith (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978368#comment-14978368
 ] 

Jeff Griffith commented on CASSANDRA-10515:
---

Great info, thanks [~benedict] ! I guess my "all roads lead to Rome" analogy 
was a good one :-)

[~blambov] i have the rest running now. normally happens a couple of times a 
day so i should know by this evening.

On that first form of growth, I unfortunately did not get a chance to try it 
before the problem corrected itself. I believe that gradually our 3 remaining 
problematic nodes too longer to reduce the # of L0 files than did the rest of 
our clusters. It took weeks rather than days and coincidentally I got involved 
near the end. From what saw, though, the symptoms seemed to match exactly what 
[~krummas] described.

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: commitlog, triage
> Attachments: C5commitLogIncrease.jpg, CommitLogProblem.jpg, 
> CommitLogSize.jpg, MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, 
> cassandra.yaml, cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Jeff Griffith (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978368#comment-14978368
 ] 

Jeff Griffith edited comment on CASSANDRA-10515 at 10/28/15 12:58 PM:
--

Great info, thanks [~benedict] ! I guess my "all roads lead to Rome" analogy 
was a good one :-)

[~blambov] i have the test running now. normally happens a couple of times a 
day so i should know by this evening.

On that first form of growth, I unfortunately did not get a chance to try it 
before the problem corrected itself. I believe that gradually our 3 remaining 
problematic nodes too longer to reduce the # of L0 files than did the rest of 
our clusters. It took weeks rather than days and coincidentally I got involved 
near the end. From what saw, though, the symptoms seemed to match exactly what 
[~krummas] described.


was (Author: jeffery.griffith):
Great info, thanks [~benedict] ! I guess my "all roads lead to Rome" analogy 
was a good one :-)

[~blambov] i have the rest running now. normally happens a couple of times a 
day so i should know by this evening.

On that first form of growth, I unfortunately did not get a chance to try it 
before the problem corrected itself. I believe that gradually our 3 remaining 
problematic nodes too longer to reduce the # of L0 files than did the rest of 
our clusters. It took weeks rather than days and coincidentally I got involved 
near the end. From what saw, though, the symptoms seemed to match exactly what 
[~krummas] described.

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: commitlog, triage
> Attachments: C5commitLogIncrease.jpg, CommitLogProblem.jpg, 
> CommitLogSize.jpg, MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, 
> cassandra.yaml, cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Jeff Griffith (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978368#comment-14978368
 ] 

Jeff Griffith edited comment on CASSANDRA-10515 at 10/28/15 1:00 PM:
-

Great info, thanks [~benedict] ! I guess my "all roads lead to Rome" analogy 
was a good one :-)

[~blambov] i have the test running now. normally happens a couple of times a 
day so i should know by this evening.

On that first form of growth, I unfortunately did not get a chance to try it 
before the problem corrected itself. I believe that gradually our 3 remaining 
problematic nodes too longer to reduce the # of L0 files than did the rest of 
our clusters. It took weeks rather than days and coincidentally I got involved 
near the end. From what I saw, though, the symptoms seemed to match exactly 
what [~krummas] described.


was (Author: jeffery.griffith):
Great info, thanks [~benedict] ! I guess my "all roads lead to Rome" analogy 
was a good one :-)

[~blambov] i have the test running now. normally happens a couple of times a 
day so i should know by this evening.

On that first form of growth, I unfortunately did not get a chance to try it 
before the problem corrected itself. I believe that gradually our 3 remaining 
problematic nodes too longer to reduce the # of L0 files than did the rest of 
our clusters. It took weeks rather than days and coincidentally I got involved 
near the end. From what saw, though, the symptoms seemed to match exactly what 
[~krummas] described.

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: commitlog, triage
> Attachments: C5commitLogIncrease.jpg, CommitLogProblem.jpg, 
> CommitLogSize.jpg, MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, 
> cassandra.yaml, cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10559) Support encrypted and plain traffic on the same port

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978280#comment-14978280
 ] 

Robert Stupp commented on CASSANDRA-10559:
--

Patch (2.1) looks fine.
dtest PR has been updated to run against 2.1, 2.2, 3.0, trunk and works fine.
Just waiting for cassci results against 2.1 + 2.2

> Support encrypted and plain traffic on the same port
> 
>
> Key: CASSANDRA-10559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10559
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Fix For: 3.0.0
>
> Attachments: 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-2.1.patch, 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-na.patch
>
>
> To be able to migrate clusters in a rolling way from plain to encrypted 
> traffic it would be very helpful if we could have Cassandra accept both on 
> the same port. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10559) Support encrypted and plain traffic on the same port

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978272#comment-14978272
 ] 

Robert Stupp edited comment on CASSANDRA-10559 at 10/28/15 11:40 AM:
-

Being able to configure different ports for SSL and non-SSL clients might be an 
operational requirement.

Unfortunately MS still uses old, blocking I/O - there's CASSANDRA-8457 to 
change that.


was (Author: snazy):
To be able to configure different ports for SSL and non-SSL clients might be an 
operational requirement.

Unfortunately MS still uses old, blocking I/O - there's CASSANDRA-8457 to 
change that.

> Support encrypted and plain traffic on the same port
> 
>
> Key: CASSANDRA-10559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10559
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Fix For: 3.0.0
>
> Attachments: 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-2.1.patch, 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-na.patch
>
>
> To be able to migrate clusters in a rolling way from plain to encrypted 
> traffic it would be very helpful if we could have Cassandra accept both on 
> the same port. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978389#comment-14978389
 ] 

Benedict commented on CASSANDRA-10089:
--

The happens before edge is guaranteed across keys. If you were to perform your 
lookups against the original map only, everything would be fine. However, 
iteration is not atomic, and the order is random. If your iteration has passed 
the position in the map for the first key _before_ it is added, you will not 
see it. However if the second key is then added, and occurs after the point you 
are at, you _will_ see it.

I suspect CHM's hash simply guarantees the order of occurrence in the map _for 
this test only, by chance_ puts the second key first. Thus guaranteeing the 
test passes.

> NullPointerException in Gossip handleStateNormal
> 
>
> Key: CASSANDRA-10089
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10089
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.1
>
> Attachments: node1_debug.log, node2_debug.log, node3_debug.log
>
>
> Whilst comparing dtests for CASSANDRA-9970 I found [this failing 
> dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/]
>  in 2.2:
> {code}
> Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 
> 15:39:57,873 CassandraDaemon.java:183 - Exception in thread 
> Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat 
> org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>  ~[main/:na] \tat 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
> ~[main/:na] \tat 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_80] \tat 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]']
> {code}
> I wasn't able to find it on unpatched branches  but it is clearly not related 
> to CASSANDRA-9970, if anything it could have been a side effect of 
> CASSANDRA-9871.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/4] cassandra git commit: Support encrypted and plain traffic on the same port

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7297b3ca2 -> 30d5b6543


Support encrypted and plain traffic on the same port

patch by Norman Maurer; reviewed by Robert Stupp for CASSANDRA-10559


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32a6f205
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32a6f205
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32a6f205

Branch: refs/heads/trunk
Commit: 32a6f2059aa3888fc0223cabcdff2ea2c9b97b21
Parents: cedcf07
Author: Norman Maurer 
Authored: Wed Oct 28 14:17:18 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:17:18 2015 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  2 +
 conf/cassandra.yaml |  2 +
 .../cassandra/config/EncryptionOptions.java |  1 +
 .../org/apache/cassandra/transport/Server.java  | 76 ++--
 5 files changed, 75 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5b46eac..998dd22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
  * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 67a545b..c6ea6c0 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,6 +18,8 @@ using the provided 'sstableupgrade' tool.
 
 New features
 
+- Native protocol server now allows both SSL and non-SSL connections on
+  the same port.
 - Switching racks is no longer an allowed operation on a node which has
   data. Instead, the node will need to be decommissioned and 
rebootstrapped.
   If moving from the SimpleSnitch, make sure the rack containing all 
current

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index e0ef878..0d0282b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -781,6 +781,8 @@ server_encryption_options:
 # enable or disable client/server encryption.
 client_encryption_options:
 enabled: false
+# If enabled and optional is set to true encrypted and unencrypted 
connections are handled.
+optional: false
 keystore: conf/.keystore
 keystore_password: cassandra
 # require_client_auth: false

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/config/EncryptionOptions.java
--
diff --git a/src/java/org/apache/cassandra/config/EncryptionOptions.java 
b/src/java/org/apache/cassandra/config/EncryptionOptions.java
index 945a15b..31f8b4a 100644
--- a/src/java/org/apache/cassandra/config/EncryptionOptions.java
+++ b/src/java/org/apache/cassandra/config/EncryptionOptions.java
@@ -36,6 +36,7 @@ public abstract class EncryptionOptions
 public static class ClientEncryptionOptions extends EncryptionOptions
 {
 public boolean enabled = false;
+public boolean optional = false;
 }
 
 public static class ServerEncryptionOptions extends EncryptionOptions

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32a6f205/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index c21a669..02f17b0 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -22,6 +22,7 @@ import java.net.InetAddress;
 import java.net.InetSocketAddress;
 import java.net.UnknownHostException;
 import java.util.EnumMap;
+import java.util.List;
 import java.util.Map;
 import java.util.concurrent.Callable;
 import java.util.concurrent.ConcurrentHashMap;
@@ -29,9 +30,11 @@ import java.util.concurrent.atomic.AtomicBoolean;
 import javax.net.ssl.SSLContext;
 import javax.net.ssl.SSLEngine;
 
+import io.netty.buffer.ByteBuf;
 import io.netty.channel.epoll.Epoll;
 import io.netty.channel.epoll.EpollEventLoopGroup;
 import io.netty.channel.epoll.EpollServerSocketChannel;
+import 

[jira] [Resolved] (CASSANDRA-10559) Support encrypted and plain traffic on the same port

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp resolved CASSANDRA-10559.
--
   Resolution: Fixed
Fix Version/s: 2.1.12
   2.2.4

CI looks good.
Committed as 32a6f2059aa3888fc0223cabcdff2ea2c9b97b21 to 2.1 + 2.2.
Thanks!

> Support encrypted and plain traffic on the same port
> 
>
> Key: CASSANDRA-10559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10559
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Fix For: 2.2.4, 2.1.12, 3.0.0
>
> Attachments: 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-2.1.patch, 
> 0001-CASSANDRA-8803-Allow-to-serve-plain-and-encrypted-na.patch
>
>
> To be able to migrate clusters in a rolling way from plain to encrypted 
> traffic it would be very helpful if we could have Cassandra accept both on 
> the same port. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/4] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[snazy]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/69037275
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/69037275
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/69037275

Branch: refs/heads/trunk
Commit: 69037275568d6fd753ee2ee40e7a5ac1751f459b
Parents: 13e9fd5 734a3bf
Author: Robert Stupp 
Authored: Wed Oct 28 14:20:02 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:20:02 2015 +0100

--
 CHANGES.txt | 2 +-
 NEWS.txt| 5 +
 2 files changed, 6 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/69037275/CHANGES.txt
--
diff --cc CHANGES.txt
index 76b1a55,c4f606f..b4cc43b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,13 +1,20 @@@
 -2.2.4
 +3.0
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
-  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
   * Do STCS in DTCS windows (CASSANDRA-10276)
 - * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)
   * Fix paging issues with partitions containing only static columns data 
(CASSANDRA-10381)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/69037275/NEWS.txt
--

[4/4] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[snazy]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30d5b654
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30d5b654
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30d5b654

Branch: refs/heads/trunk
Commit: 30d5b6543fc00b56d2940470327c9242415e7951
Parents: 7297b3c 6903727
Author: Robert Stupp 
Authored: Wed Oct 28 14:21:01 2015 +0100
Committer: Robert Stupp 
Committed: Wed Oct 28 14:21:01 2015 +0100

--
 CHANGES.txt | 2 +-
 NEWS.txt| 5 +
 2 files changed, 6 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30d5b654/CHANGES.txt
--

[1/3] cassandra git commit: Correctly mark indexes built/removed when rebuilding

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 5bc3c4b6d -> 13e9fd55f
  refs/heads/trunk 81183d6e2 -> 7297b3ca2


Correctly mark indexes built/removed when rebuilding

Patch by Sam Tunnicliffe; reviewed by Benjamin Lerer for CASSANDRA-10601


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13e9fd55
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13e9fd55
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13e9fd55

Branch: refs/heads/cassandra-3.0
Commit: 13e9fd55ff93c26ca339dfa30ed9ac7113894e8e
Parents: 5bc3c4b
Author: Sam Tunnicliffe 
Authored: Tue Oct 27 15:09:18 2015 +
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 11:36:44 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/index/SecondaryIndexManager.java  | 11 +++---
 .../index/internal/CassandraIndex.java  | 15 ++--
 .../index/internal/CassandraIndexTest.java  | 20 +++
 .../index/internal/CustomCassandraIndex.java| 38 ++--
 5 files changed, 31 insertions(+), 54 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13e9fd55/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index dd7949e..76b1a55 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix marking of indexes as built and removed (CASSANDRA-10601)
  * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
  * Fix batches on multiple tables (CASSANDRA-10554)
  * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13e9fd55/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --git a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java 
b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 7d866df..3ed9714 100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@ -185,8 +185,8 @@ public class SecondaryIndexManager implements IndexRegistry
 Index index = indexes.remove(indexName);
 if (null != index)
 {
+markIndexRemoved(indexName);
 executeBlocking(index.getInvalidateTask());
-unregisterIndex(index);
 }
 }
 
@@ -355,14 +355,14 @@ public class SecondaryIndexManager implements 
IndexRegistry
 indexes.stream().map(i -> 
i.getIndexMetadata().name).collect(Collectors.joining(",")));
 }
 
-private void markIndexBuilt(String indexName)
+public void markIndexBuilt(String indexName)
 {
-SystemKeyspace.setIndexBuilt(baseCfs.name, indexName);
+SystemKeyspace.setIndexBuilt(baseCfs.keyspace.getName(), indexName);
 }
 
-private void markIndexRemoved(String indexName)
+public void markIndexRemoved(String indexName)
 {
-SystemKeyspace.setIndexRemoved(baseCfs.name, indexName);
+SystemKeyspace.setIndexRemoved(baseCfs.keyspace.getName(), indexName);
 }
 
 
@@ -410,6 +410,7 @@ public class SecondaryIndexManager implements IndexRegistry
  */
 public void invalidateAllIndexesBlocking()
 {
+markAllIndexesRemoved();
 executeAllBlocking(indexes.values().stream(), 
Index::getInvalidateTask);
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13e9fd55/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
--
diff --git a/src/java/org/apache/cassandra/index/internal/CassandraIndex.java 
b/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
index 7b9d18f..5b5b7e4 100644
--- a/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
+++ b/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
@@ -176,7 +176,6 @@ public abstract class CassandraIndex implements Index
 public Callable getInvalidateTask()
 {
 return () -> {
-markRemoved();
 invalidate();
 return null;
 };
@@ -632,16 +631,6 @@ public abstract class CassandraIndex implements Index
 return SystemKeyspace.isIndexBuilt(baseCfs.keyspace.getName(), 
metadata.name);
 }
 
-private void markBuilt()
-{
-SystemKeyspace.setIndexBuilt(baseCfs.keyspace.getName(), 
metadata.name);
-}
-
-private void markRemoved()
-{
-SystemKeyspace.setIndexRemoved(baseCfs.keyspace.getName(), 
metadata.name);
-}
-
 private boolean isPrimaryKeyIndex()
 {
 

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[samt]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7297b3ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7297b3ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7297b3ca

Branch: refs/heads/trunk
Commit: 7297b3ca22c13369b494c63a80f20ae0379f3fbf
Parents: 81183d6 13e9fd5
Author: Sam Tunnicliffe 
Authored: Wed Oct 28 11:37:08 2015 +
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 11:37:08 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/index/SecondaryIndexManager.java  | 11 +++---
 .../index/internal/CassandraIndex.java  | 15 ++--
 .../index/internal/CassandraIndexTest.java  | 20 +++
 .../index/internal/CustomCassandraIndex.java| 38 ++--
 5 files changed, 31 insertions(+), 54 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7297b3ca/CHANGES.txt
--
diff --cc CHANGES.txt
index 238d6b6,76b1a55..b3e0349
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * Fix marking of indexes as built and removed (CASSANDRA-10601)
   * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
   * Fix batches on multiple tables (CASSANDRA-10554)
   * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7297b3ca/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
--

[2/3] cassandra git commit: Correctly mark indexes built/removed when rebuilding

[samt]

Correctly mark indexes built/removed when rebuilding

Patch by Sam Tunnicliffe; reviewed by Benjamin Lerer for CASSANDRA-10601


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13e9fd55
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13e9fd55
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13e9fd55

Branch: refs/heads/trunk
Commit: 13e9fd55ff93c26ca339dfa30ed9ac7113894e8e
Parents: 5bc3c4b
Author: Sam Tunnicliffe 
Authored: Tue Oct 27 15:09:18 2015 +
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 11:36:44 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/index/SecondaryIndexManager.java  | 11 +++---
 .../index/internal/CassandraIndex.java  | 15 ++--
 .../index/internal/CassandraIndexTest.java  | 20 +++
 .../index/internal/CustomCassandraIndex.java| 38 ++--
 5 files changed, 31 insertions(+), 54 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/13e9fd55/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index dd7949e..76b1a55 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix marking of indexes as built and removed (CASSANDRA-10601)
  * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
  * Fix batches on multiple tables (CASSANDRA-10554)
  * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13e9fd55/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --git a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java 
b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 7d866df..3ed9714 100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@ -185,8 +185,8 @@ public class SecondaryIndexManager implements IndexRegistry
 Index index = indexes.remove(indexName);
 if (null != index)
 {
+markIndexRemoved(indexName);
 executeBlocking(index.getInvalidateTask());
-unregisterIndex(index);
 }
 }
 
@@ -355,14 +355,14 @@ public class SecondaryIndexManager implements 
IndexRegistry
 indexes.stream().map(i -> 
i.getIndexMetadata().name).collect(Collectors.joining(",")));
 }
 
-private void markIndexBuilt(String indexName)
+public void markIndexBuilt(String indexName)
 {
-SystemKeyspace.setIndexBuilt(baseCfs.name, indexName);
+SystemKeyspace.setIndexBuilt(baseCfs.keyspace.getName(), indexName);
 }
 
-private void markIndexRemoved(String indexName)
+public void markIndexRemoved(String indexName)
 {
-SystemKeyspace.setIndexRemoved(baseCfs.name, indexName);
+SystemKeyspace.setIndexRemoved(baseCfs.keyspace.getName(), indexName);
 }
 
 
@@ -410,6 +410,7 @@ public class SecondaryIndexManager implements IndexRegistry
  */
 public void invalidateAllIndexesBlocking()
 {
+markAllIndexesRemoved();
 executeAllBlocking(indexes.values().stream(), 
Index::getInvalidateTask);
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/13e9fd55/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
--
diff --git a/src/java/org/apache/cassandra/index/internal/CassandraIndex.java 
b/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
index 7b9d18f..5b5b7e4 100644
--- a/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
+++ b/src/java/org/apache/cassandra/index/internal/CassandraIndex.java
@@ -176,7 +176,6 @@ public abstract class CassandraIndex implements Index
 public Callable getInvalidateTask()
 {
 return () -> {
-markRemoved();
 invalidate();
 return null;
 };
@@ -632,16 +631,6 @@ public abstract class CassandraIndex implements Index
 return SystemKeyspace.isIndexBuilt(baseCfs.keyspace.getName(), 
metadata.name);
 }
 
-private void markBuilt()
-{
-SystemKeyspace.setIndexBuilt(baseCfs.keyspace.getName(), 
metadata.name);
-}
-
-private void markRemoved()
-{
-SystemKeyspace.setIndexRemoved(baseCfs.keyspace.getName(), 
metadata.name);
-}
-
 private boolean isPrimaryKeyIndex()
 {
 return indexedColumn.isPrimaryKeyColumn();
@@ -668,7 +657,7 @@ public abstract class CassandraIndex implements Index
 

[jira] [Reopened] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Alan Boudreault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Boudreault reopened CASSANDRA-9664:

Reproduced In: 3.0.0 rc1

Reopening this issue. This change caused an important regression with MV 
performance. I'm using mvbench to test with RF=3 and I confirm it's not a 
driver issue. 

{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint node1"

3.0.0-beta2 (alpha2 java driver)
---
total
 count = 461601
 mean rate = 1923.21 calls/second
 1-minute rate = 1937.82 calls/second
 5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
   min = 1.90 milliseconds
   max = 3707.76 milliseconds
  mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
  75% <= 775.95 milliseconds
  95% <= 1417.67 milliseconds
  98% <= 1728.05 milliseconds
  99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds


3.0.0-rc1 (alpha3 java driver)
-

total
 count = 310373
 mean rate = 272.25 calls/second
 1-minute rate = 0.00 calls/second
 5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
   min = 1.05 milliseconds
   max = 10468.98 milliseconds
  mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
  75% <= 696.25 milliseconds
  95% <= 1434.45 milliseconds
  98% <= 1820.33 milliseconds
  99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}

[~thobbs] any idea what could be wrong?

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10611) Upgrade test on 2.1->3.0 path fails with configuration problems

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10611:


 Summary: Upgrade test on 2.1->3.0 path fails with configuration 
problems
 Key: CASSANDRA-10611
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10611
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
Assignee: Russ Hatch


The following test fails on the uprgrade path from 2.1 to 3.0:

http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/upgrade_through_versions_test/TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD/bootstrap_multidc_test/

I believe it's basically a configuration error; the cluster likely just needs 
to be reconfigured in the test:

{code}
code=2200 [Invalid query] message="User-defined functions are disabled in 
cassandra.yaml - set enable_user_defined_functions=true to enable"
{code}

Assigning to [~rhatch] for now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock

[Mike Adamson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Adamson updated CASSANDRA-10608:
-
Attachment: 10608.txt

> Adding a dynamic column to a compact storage table with the same name as the 
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows: 
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy', 
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column, 
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}} 
> containing a {{PartitionColumns}} containing the partition key 
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether 
> the {{ColumnDefinition}} is a partition key 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[slebresne]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9be01e34
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9be01e34
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9be01e34

Branch: refs/heads/trunk
Commit: 9be01e34c8adaa9558458e864969d22b05cb8e87
Parents: 30d5b65 4d00468
Author: Sylvain Lebresne 
Authored: Wed Oct 28 16:14:10 2015 +0100
Committer: Sylvain Lebresne 
Committed: Wed Oct 28 16:14:10 2015 +0100

--
 CHANGES.txt|  1 +
 .../apache/cassandra/thrift/CassandraServer.java   | 17 +++--
 2 files changed, 16 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9be01e34/CHANGES.txt
--
diff --cc CHANGES.txt
index 5f3abbe,3423a1e..04cefae
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * Fix thrift get() queries with defined columns (CASSANDRA-10586)
   * Fix marking of indexes as built and removed (CASSANDRA-10601)
   * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
   * Fix batches on multiple tables (CASSANDRA-10554)

[jira] [Comment Edited] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Alan Boudreault (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978456#comment-14978456
 ] 

Alan Boudreault edited comment on CASSANDRA-9664 at 10/28/15 1:59 PM:
--

Reopening this issue. This change caused an important regression with MV 
performance. I'm using mvbench to test with RF=3 and I confirm it's not a 
driver issue. 

{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint node1"

3.0.0-beta2 (alpha2 java driver)
---
total
 count = 461601
 mean rate = 1923.21 calls/second
 1-minute rate = 1937.82 calls/second
 5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
   min = 1.90 milliseconds
   max = 3707.76 milliseconds
  mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
  75% <= 775.95 milliseconds
  95% <= 1417.67 milliseconds
  98% <= 1728.05 milliseconds
  99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds


3.0.0-rc1 (alpha3 java driver)
-

total
 count = 310373
 mean rate = 272.25 calls/second
 1-minute rate = 0.00 calls/second
 5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
   min = 1.05 milliseconds
   max = 10468.98 milliseconds
  mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
  75% <= 696.25 milliseconds
  95% <= 1434.45 milliseconds
  98% <= 1820.33 milliseconds
  99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}

[~thobbs] any idea what could make things getting slower?


was (Author: aboudreault):
Reopening this issue. This change caused an important regression with MV 
performance. I'm using mvbench to test with RF=3 and I confirm it's not a 
driver issue. 

{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint node1"

3.0.0-beta2 (alpha2 java driver)
---
total
 count = 461601
 mean rate = 1923.21 calls/second
 1-minute rate = 1937.82 calls/second
 5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
   min = 1.90 milliseconds
   max = 3707.76 milliseconds
  mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
  75% <= 775.95 milliseconds
  95% <= 1417.67 milliseconds
  98% <= 1728.05 milliseconds
  99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds


3.0.0-rc1 (alpha3 java driver)
-

total
 count = 310373
 mean rate = 272.25 calls/second
 1-minute rate = 0.00 calls/second
 5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
   min = 1.05 milliseconds
   max = 10468.98 milliseconds
  mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
  75% <= 696.25 milliseconds
  95% <= 1434.45 milliseconds
  98% <= 1820.33 milliseconds
  99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}

[~thobbs] any idea what could be wrong?

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8965) Cassandra retains a file handle to the directory its writing to for each writer instance

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8965:

Fix Version/s: 2.2.x
   2.1.x

> Cassandra retains a file handle to the directory its writing to for each 
> writer instance
> 
>
> Key: CASSANDRA-8965
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8965
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Priority: Trivial
> Fix For: 2.1.x, 2.2.x, 3.1
>
>
> We could either share this amongst the CF object, or have a shared 
> ref-counted cache that opens a reference and shares it amongst all writer 
> instances, closing it once they all close.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Alan Boudreault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Boudreault resolved CASSANDRA-9664.

Resolution: Fixed

Closing in favor of CASSANDRA-10609.

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix thrift get() when reading regular columns

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 690372755 -> 4d00468cc


Fix thrift get() when reading regular columns

patch by slebresne; reviewed by iamaleksey for CASSANDRA-10586


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d00468c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d00468c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d00468c

Branch: refs/heads/cassandra-3.0
Commit: 4d00468cc38014ed1bd7f9da93ef3a339682f7c1
Parents: 6903727
Author: Sylvain Lebresne 
Authored: Tue Oct 27 14:46:56 2015 +0100
Committer: Sylvain Lebresne 
Committed: Wed Oct 28 16:10:51 2015 +0100

--
 CHANGES.txt|  1 +
 .../apache/cassandra/thrift/CassandraServer.java   | 17 +++--
 2 files changed, 16 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d00468c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b4cc43b..3423a1e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix thrift get() queries with defined columns (CASSANDRA-10586)
  * Fix marking of indexes as built and removed (CASSANDRA-10601)
  * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
  * Fix batches on multiple tables (CASSANDRA-10554)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d00468c/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index 538d128..2a3f73d 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -623,8 +623,21 @@ public class CassandraServer implements Cassandra.Iface
 else
 {
 LegacyLayout.LegacyCellName cellname = 
LegacyLayout.decodeCellName(metadata, column_path.super_column, 
column_path.column);
-columns = 
ColumnFilter.selection(PartitionColumns.of(cellname.column));
-filter = new 
ClusteringIndexNamesFilter(FBUtilities.singleton(cellname.clustering, 
metadata.comparator), false);
+if (cellname.clustering == Clustering.STATIC_CLUSTERING)
+{
+// Same as above: even if we're querying a static column, 
we still query the equivalent dynamic column and value as some
+// values might have been created post creation of the 
column (ThriftResultMerger then ensures we get only one result).
+ColumnFilter.Builder builder = 
ColumnFilter.selectionBuilder();
+builder.add(cellname.column);
+builder.add(metadata.compactValueColumn());
+columns = builder.build();
+filter = new 
ClusteringIndexNamesFilter(FBUtilities.singleton(new 
Clustering(column_path.column), metadata.comparator), false);
+}
+else
+{
+columns = 
ColumnFilter.selection(PartitionColumns.of(cellname.column));
+filter = new 
ClusteringIndexNamesFilter(FBUtilities.singleton(cellname.clustering, 
metadata.comparator), false);
+}
 }
 
 DecoratedKey dk = metadata.decorateKey(key);

[1/2] cassandra git commit: Fix thrift get() when reading regular columns

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 30d5b6543 -> 9be01e34c


Fix thrift get() when reading regular columns

patch by slebresne; reviewed by iamaleksey for CASSANDRA-10586


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d00468c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d00468c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d00468c

Branch: refs/heads/trunk
Commit: 4d00468cc38014ed1bd7f9da93ef3a339682f7c1
Parents: 6903727
Author: Sylvain Lebresne 
Authored: Tue Oct 27 14:46:56 2015 +0100
Committer: Sylvain Lebresne 
Committed: Wed Oct 28 16:10:51 2015 +0100

--
 CHANGES.txt|  1 +
 .../apache/cassandra/thrift/CassandraServer.java   | 17 +++--
 2 files changed, 16 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d00468c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b4cc43b..3423a1e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix thrift get() queries with defined columns (CASSANDRA-10586)
  * Fix marking of indexes as built and removed (CASSANDRA-10601)
  * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
  * Fix batches on multiple tables (CASSANDRA-10554)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d00468c/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index 538d128..2a3f73d 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -623,8 +623,21 @@ public class CassandraServer implements Cassandra.Iface
 else
 {
 LegacyLayout.LegacyCellName cellname = 
LegacyLayout.decodeCellName(metadata, column_path.super_column, 
column_path.column);
-columns = 
ColumnFilter.selection(PartitionColumns.of(cellname.column));
-filter = new 
ClusteringIndexNamesFilter(FBUtilities.singleton(cellname.clustering, 
metadata.comparator), false);
+if (cellname.clustering == Clustering.STATIC_CLUSTERING)
+{
+// Same as above: even if we're querying a static column, 
we still query the equivalent dynamic column and value as some
+// values might have been created post creation of the 
column (ThriftResultMerger then ensures we get only one result).
+ColumnFilter.Builder builder = 
ColumnFilter.selectionBuilder();
+builder.add(cellname.column);
+builder.add(metadata.compactValueColumn());
+columns = builder.build();
+filter = new 
ClusteringIndexNamesFilter(FBUtilities.singleton(new 
Clustering(column_path.column), metadata.comparator), false);
+}
+else
+{
+columns = 
ColumnFilter.selection(PartitionColumns.of(cellname.column));
+filter = new 
ClusteringIndexNamesFilter(FBUtilities.singleton(cellname.clustering, 
metadata.comparator), false);
+}
 }
 
 DecoratedKey dk = metadata.decorateKey(key);

[jira] [Updated] (CASSANDRA-10606) PartitionUpdate.operationCount() return the wrong number of operation for compact tables

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10606:

Reviewer: Sylvain Lebresne

> PartitionUpdate.operationCount() return the wrong number of operation for 
> compact tables
> 
>
> Key: CASSANDRA-10606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10606
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.0.0
>
> Attachments: 10606-3.0-V2.txt, 10606-3.0-V3.txt, 10606-3.0.txt
>
>
> For compact tables {{PartitionUpdate.operationCount()}} return the wrong 
> number of operation as it does not take into operation on static rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978469#comment-14978469
 ] 

Aleksey Yeschenko commented on CASSANDRA-9664:
--

Can you open a new ticket instead? In general we try not to reopen tickets that 
have been included in a released C* version.

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10590) LegacySSTableTest fails after CASSANDRA-10360

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978473#comment-14978473
 ] 

Sylvain Lebresne commented on CASSANDRA-10590:
--

That's definitively my bad for missing that test, sorry. The issue has to do 
with the fact that the legacy deserializer {{hasNext()}} method deserialize the 
full row upfront, and the fact that a row can span multiple index block (again, 
in the legacy case). The patch for CASSANDRA-10360 accounted for that but 
missed one part, so that we were potentially stopping reading early because a 
call to {{hasNext()}} put us on an index block where we knew not row we were 
interested in could start, but that made us mistakenly skip the next row (the 
one read in the {{hasNext()}} call). Anyway, pushed a patch to fix this 
[here|https://github.com/pcmanus/cassandra/commits/10590].

> LegacySSTableTest fails after CASSANDRA-10360
> -
>
> Key: CASSANDRA-10590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10590
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0
>
>
> LegacySSTableTest fails reading pre-3.0 sstables (versions {{jb}}, {{ka}}, 
> {{la}}) with clustering keys and counters.
> [First failing 3.0 testall 
> build|http://cassci.datastax.com/job/cassandra-3.0_testall/205/]
> /cc [~slebresne]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10604) Secondary index metadata is not reloaded when table is altered

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-10604:

Reviewer: Tyler Hobbs

> Secondary index metadata is not reloaded when table is altered
> --
>
> Key: CASSANDRA-10604
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10604
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> The javadocs for {{Index.getMetadataReloadTask()}} state the following:
> {quote}
> Returns a task to reload the internal metadata of an index.
> Called when the base table metadata is modified or when the configuration of 
> the Index is updated.
> {quote}
> However, altering a table does not result in the reload task being executed.  
> I think the root of the problem is that in 
> {{SecondaryIndexManager.reloadIndex()}}, we only execute the reload task when 
> the old {{IndexMetadata}} does not equal the current {{IndexMetadata}}.  
> Altering the table does not change the index metadata, so this check always 
> fails.
> This especially affects per-row secondary indexes, where the index may need 
> to handle columns being added or dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10513) Update cqlsh for new driver execution API

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978485#comment-14978485
 ] 

Paulo Motta commented on CASSANDRA-10513:
-

* Added new 
[commit|https://github.com/apache/cassandra/commit/c3b58a2925780388d3ee7b9db30e3792edb2351b]
 fixing table metadata access on {{cqlsh.py}}.
* Created cassandra-dtest 
[PR|https://github.com/riptano/cassandra-dtest/pull/631] with simple fix on 
index tests.
* Submitted new test runs:
||2.2||3.0||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-10513]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-10513]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-10513]|
|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10513-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10513-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10513-dtest/lastCompletedBuild/testReport/]|

> Update cqlsh for new driver execution API
> -
>
> Key: CASSANDRA-10513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10513
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Adam Holmberg
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.2.x, 3.0.x
>
> Attachments: 10513-2.1.txt, 10513-2.2.txt, 10513.txt
>
>
> The 3.0 Python driver will have a few tweaks to the execution API. We'll need 
> to update cqlsh in a couple of minor ways.
> [Results are always returned as an iterable 
> ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-368]
> [Trace data is always attached to the 
> ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-318] (instead of 
> being attached to a statement)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978507#comment-14978507
 ] 

Aleksey Yeschenko commented on CASSANDRA-10365:
---

Force-pushed updates to the same branch.

Everything besides UDTs that refer to other UDTs should work now (except maybe 
certain parts of UDFs/UDAs, but I need the driver fixed first to verify them).

Otherwise the last remaining bits is cross-UDT dependencies. Finalising that, 
will push soon-ish.

> Consider storing types by their CQL names in schema tables instead of 
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10613) Upgrade test on 2.1->3.0 path fails with NPE in getExistingFiles (likely known bug)

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10613:


 Summary: Upgrade test on 2.1->3.0 path fails with NPE in 
getExistingFiles (likely known bug)
 Key: CASSANDRA-10613
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10613
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
Assignee: Russ Hatch


In this job:

http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/

The following tests fail due to an NPE in 
{{org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles}}:

upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.bootstrap_test
upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.rolling_upgrade_test
upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.parallel_upgrade_with_internode_ssl_test
upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.rolling_upgrade_with_internode_ssl_test
upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD.rolling_upgrade_with_internode_ssl_test
upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.parallel_upgrade_test

I believe this is likely happening because of CASSANDRA-10602, so let's hold 
off on messing with this until that's merged.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10576) Thrift CAS on static columns doesn't work as expected

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978604#comment-14978604
 ] 

Sylvain Lebresne commented on CASSANDRA-10576:
--

+1

Nit: in {{ThriftCasRequest}}, there is 2 places where we do {{if (expected == 
null || expected.isEmpty())}}: we can ditch the {{expected == null}} now. 

> Thrift CAS on static columns doesn't work as expected
> -
>
> Key: CASSANDRA-10576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10576
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.0
>
>
> Although the thrift cas call works as expected for dynamic column families, 
> using it on tables with statically defined columns produces unexpected 
> results. The problem, in {{ThriftCASRequest}}, is that while the {{expected}} 
> partition contains a static row, the {{current}} partition has been processed 
> by {{ThriftResultsMerger}} during the read, converting the static columns to 
> clusterings. If {{expected}} contains only a static row, no further checking 
> is carried out, {{appliesTo}} erroneously returns true and the conditional 
> update is performed regardless of the current state of the partition.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10609) MV performance regression

[Alan Boudreault (JIRA)]

Alan Boudreault created CASSANDRA-10609:
---

 Summary: MV performance regression
 Key: CASSANDRA-10609
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10609
 Project: Cassandra
  Issue Type: Bug
 Environment: EC2
Reporter: Alan Boudreault
Assignee: Tyler Hobbs
Priority: Critical
 Fix For: 3.0.0


I've noticed an important MV performance regression that has been introduced in 
3.0.0-rc1. The issue has been introduced by #9664.

* I'm using mvbench to test with RF=3
* I confirm it's not a driver issue.

{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint node1"

3.0.0-beta2 (alpha2 java driver)
---
total
 count = 461601
 mean rate = 1923.21 calls/second
 1-minute rate = 1937.82 calls/second
 5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
   min = 1.90 milliseconds
   max = 3707.76 milliseconds
  mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
  75% <= 775.95 milliseconds
  95% <= 1417.67 milliseconds
  98% <= 1728.05 milliseconds
  99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds


3.0.0-rc1 (alpha3 java driver)
-

total
 count = 310373
 mean rate = 272.25 calls/second
 1-minute rate = 0.00 calls/second
 5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
   min = 1.05 milliseconds
   max = 10468.98 milliseconds
  mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
  75% <= 696.25 milliseconds
  95% <= 1434.45 milliseconds
  98% <= 1820.33 milliseconds
  99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10609) MV performance regression

[Alan Boudreault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Boudreault updated CASSANDRA-10609:

Description: 
I've noticed an important MV performance regression that has been introduced in 
3.0.0-rc1. The issue has been introduced by CASSANDRA-9664.

* I'm using mvbench to test with RF=3
* I confirm it's not a driver issue.

{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint node1"

3.0.0-beta2 (alpha2 java driver)
---
total
 count = 461601
 mean rate = 1923.21 calls/second
 1-minute rate = 1937.82 calls/second
 5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
   min = 1.90 milliseconds
   max = 3707.76 milliseconds
  mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
  75% <= 775.95 milliseconds
  95% <= 1417.67 milliseconds
  98% <= 1728.05 milliseconds
  99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds


3.0.0-rc1 (alpha3 java driver)
-

total
 count = 310373
 mean rate = 272.25 calls/second
 1-minute rate = 0.00 calls/second
 5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
   min = 1.05 milliseconds
   max = 10468.98 milliseconds
  mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
  75% <= 696.25 milliseconds
  95% <= 1434.45 milliseconds
  98% <= 1820.33 milliseconds
  99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}

  was:
I've noticed an important MV performance regression that has been introduced in 
3.0.0-rc1. The issue has been introduced by #9664.

* I'm using mvbench to test with RF=3
* I confirm it's not a driver issue.

{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint node1"

3.0.0-beta2 (alpha2 java driver)
---
total
 count = 461601
 mean rate = 1923.21 calls/second
 1-minute rate = 1937.82 calls/second
 5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
   min = 1.90 milliseconds
   max = 3707.76 milliseconds
  mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
  75% <= 775.95 milliseconds
  95% <= 1417.67 milliseconds
  98% <= 1728.05 milliseconds
  99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds


3.0.0-rc1 (alpha3 java driver)
-

total
 count = 310373
 mean rate = 272.25 calls/second
 1-minute rate = 0.00 calls/second
 5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
   min = 1.05 milliseconds
   max = 10468.98 milliseconds
  mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
  75% <= 696.25 milliseconds
  95% <= 1434.45 milliseconds
  98% <= 1820.33 milliseconds
  99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}


> MV performance regression
> -
>
> Key: CASSANDRA-10609
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10609
> Project: Cassandra
>  Issue Type: Bug
> Environment: EC2
>Reporter: Alan Boudreault
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> I've noticed an important MV performance regression that has been introduced 
> in 3.0.0-rc1. The issue has been introduced by CASSANDRA-9664.
> * I'm using mvbench to test with RF=3
> * I confirm it's not a driver issue.
> {code}
> EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint node1"
> 3.0.0-beta2 (alpha2 java driver)
> ---
> total
>  count = 461601
>  mean rate = 1923.21 calls/second
>  1-minute rate = 1937.82 calls/second
>  5-minute rate = 1424.09 calls/second
> 15-minute rate = 1058.28 calls/second
>min = 1.90 milliseconds
>max = 3707.76 milliseconds
>   mean = 516.42 milliseconds
> stddev = 457.41 milliseconds
> median = 390.07 milliseconds
>   75% <= 775.95 milliseconds
>   95% <= 

[jira] [Created] (CASSANDRA-10610) Protocol upgrade tests for bootstrapping failing on 2.1->3.0 path

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10610:


 Summary: Protocol upgrade tests for bootstrapping failing on 
2.1->3.0 path
 Key: CASSANDRA-10610
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10610
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
Assignee: Russ Hatch


This ticket is for a couple failures on protocol v3 upgrade test job on CassCI:

http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/

The failures are:

* 
[upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD.bootstrap_multidc_test|http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD/bootstrap_multidc_test/]
* 
[upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag.bootstrap_multidc_test|http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag/bootstrap_multidc_test/]

They fail with the following error:

{code}
code=1000 [Unavailable exception] message="Cannot achieve consistency level 
ALL" info={'required_replicas': 3, 'alive_replicas': 0, 'consistency': 'ALL'}
{code}

Assigning [~rhatch], since you're the most likely to understand what's going on 
here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978555#comment-14978555
 ] 

Aleksey Yeschenko commented on CASSANDRA-10365:
---

bq. Curious if you have given any thought to storing 
system_schema.aggregates.initcond as a text CQL literal. This is the only thing 
I have come across in my integration thus far that requires the client to parse 
types in order to reproduce DDL.

[~snazy] was kind enough to volunteer to deal with that bit of the ticket - 
will most likely be done tomorrow, no objections.

> Consider storing types by their CQL names in schema tables instead of 
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Alan Boudreault (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978474#comment-14978474
 ] 

Alan Boudreault commented on CASSANDRA-9664:


sure

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10608:

Reviewer: T Jake Luciani

> Adding a dynamic column to a compact storage table with the same name as the 
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows: 
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy', 
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column, 
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}} 
> containing a {{PartitionColumns}} containing the partition key 
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether 
> the {{ColumnDefinition}} is a partition key 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9223) ArithmeticException after decommission

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-9223:

Assignee: Branimir Lambov

> ArithmeticException after decommission
> --
>
> Key: CASSANDRA-9223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9223
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Branimir Lambov
>Priority: Minor
> Fix For: 3.1
>
>
> Also seen on trunk while working on CASSANDRA-8072:
> {noformat}
> ERROR 19:21:33 Exception in thread Thread[BatchlogTasks:1,5,main]
> java.lang.ArithmeticException: / by zero
>  at 
> org.apache.cassandra.db.BatchlogManager.replayAllFailedBatches(BatchlogManager.java:173)
>  ~[main/:na]
>  at 
> org.apache.cassandra.db.BatchlogManager.access$000(BatchlogManager.java:61) 
> ~[main/:na]
>  at 
> org.apache.cassandra.db.BatchlogManager$1.runMayThrow(BatchlogManager.java:91)
>  ~[main/:na]
>  at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>  at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:82)
>  ~[main/:na]
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_76]
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_76]
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_76]
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_76]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_76]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_76]
>  at java.lang.Thread.run(Thread.java:745) [na:1.7.0_76]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10586) thrift get on compact storage table broken

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978529#comment-14978529
 ] 

Aleksey Yeschenko commented on CASSANDRA-10586:
---

+1

> thrift get on compact storage table broken
> --
>
> Key: CASSANDRA-10586
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10586
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Sylvain Lebresne
>Priority: Critical
> Fix For: 3.0.0
>
>
> dtest here that triggers the issue:
> https://github.com/riptano/cassandra-dtest/pull/629
> {code}
> class TestCompactStorageThriftAccesses(ThriftTester):
> def test_get(self):
> node1, = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> # Create a CQL table with a static column and insert a row
> session.execute("USE \"Keyspace1\"");
> session.execute("CREATE TABLE IF NOT EXISTS cs1 (k int PRIMARY KEY,v 
> int) WITH COMPACT STORAGE");
> _set_keyspace('Keyspace1')
> CL = ConsistencyLevel.ONE
> i=1
> client.insert(_i32(i), ColumnParent('cs1'), Column('v', _i32(i), 0), 
> CL)
> _assert_column('cs1', _i32(i), 'v', _i32(i), 0)
> {code}
> {noformat}
> $ CASSANDRA_DIR=../cassandra nosetests 
> thrift_tests:TestCompactStorageThriftAccesses
> EE
> ==
> ERROR: test_get (thrift_tests.TestCompactStorageThriftAccesses)
> --
> Traceback (most recent call last):
>   File "/Users/jeremiah/github/cassandra-dtest/thrift_tests.py", line 2342, 
> in test_get
> _assert_column('cs1', _i32(i), 'v', _i32(i), 0)
>   File "/Users/jeremiah/github/cassandra-dtest/thrift_tests.py", line 138, in 
> _assert_column
> assert client.get(key, ColumnPath(column_family, column=column), 
> ConsistencyLevel.ONE).column == Column(column, value, ts)
>   File 
> "/Users/jeremiah/github/cassandra-dtest/thrift_bindings/v22/Cassandra.py", 
> line 585, in get
> return self.recv_get()
>   File 
> "/Users/jeremiah/github/cassandra-dtest/thrift_bindings/v22/Cassandra.py", 
> line 598, in recv_get
> (fname, mtype, rseqid) = self._iprot.readMessageBegin()
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
>  line 126, in readMessageBegin
> sz = self.readI32()
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
>  line 206, in readI32
> buff = self.trans.readAll(4)
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/transport/TTransport.py",
>  line 58, in readAll
> chunk = self.read(sz - have)
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/transport/TTransport.py",
>  line 276, in read
> self.readFrame()
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/transport/TTransport.py",
>  line 280, in readFrame
> buff = self.__trans.readAll(4)
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/transport/TTransport.py",
>  line 58, in readAll
> chunk = self.read(sz - have)
>   File 
> "/Users/jeremiah/.virtualenvs/py27/lib/python2.7/site-packages/thrift/transport/TSocket.py",
>  line 120, in read
> message='TSocket read 0 bytes')
> TTransportException: TSocket read 0 bytes
>  >> begin captured stdout << -
> [node1 ERROR] objc[10539]: Class JavaLaunchHelper is implemented in both 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home//bin/java and 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/libinstrument.dylib.
>  One of the two will be used. Which one is undefined.
> - >> end captured stdout << --
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: 
> /var/folders/t4/rlc2b6450qbg92762l9l4mt8gn/T/dtest-HtfTkF
> - >> end captured logging << -
> ==
> ERROR: test_get (thrift_tests.TestCompactStorageThriftAccesses)
> --
> Traceback (most recent call last):
>   File "/Users/jeremiah/github/cassandra-dtest/dtest.py", line 550, in 
> tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name, 
> errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [Thrift:1] 
> 2015-10-24 02:50:02,460 CassandraDaemon.java:195 - Exception in thread 
> 

[jira] [Created] (CASSANDRA-10612) Protocol v3 upgrade tests on 2.1->3.0 path fail when compaction is interrupted

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10612:


 Summary: Protocol v3 upgrade tests on 2.1->3.0 path fail when 
compaction is interrupted
 Key: CASSANDRA-10612
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10612
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
Assignee: Russ Hatch


The following tests in the upgrade_through_versions dtest suite fail:

* 
upgrade_through_versions_test.py:TestRandomPartitionerUpgrade.rolling_upgrade_test
* 
upgrade_through_versions_test.py:TestRandomPartitionerUpgrade.rolling_upgrade_with_internode_ssl_test
* 
upgrade_through_versions_test.py:TestUpgradeThroughVersions.rolling_upgrade_with_internode_ssl_test
* 
upgrade_through_versions_test.py:TestUpgradeThroughVersions.rolling_upgrade_test
* 
upgrade_through_versions_test.py:TestUpgrade_from_2_1_latest_tag_to_cassandra_3_0_HEAD.rolling_upgrade_test
* 
upgrade_through_versions_test.py:TestUpgrade_from_2_1_latest_tag_to_cassandra_3_0_HEAD.rolling_upgrade_with_internode_ssl_test
* 
upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag.rolling_upgrade_test
* 
upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag.rolling_upgrade_with_internode_ssl_test
* 
upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD.rolling_upgrade_test

See this report:

http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/

They fail with the following error:


{code}
A subprocess has terminated early. Subprocess statuses: Process-41 (is_alive: 
True), Process-42 (is_alive: False), Process-43 (is_alive: True), Process-44 
(is_alive: False), attempting to terminate remaining subprocesses now.
{code}

and with logs that look like this:

{code}
Unexpected error in node1 node log: ['ERROR [SecondaryIndexManagement:1] 
2015-10-27 00:06:52,335 CassandraDaemon.java:195 - Exception in thread 
Thread[SecondaryIndexManagement:1,5,main] java.lang.RuntimeException: 
java.util.concurrent.ExecutionException: 
org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
interrupted: Secondary index 
build@41202370-7c3e-11e5-9331-6bb6e58f8b1b(upgrade, cf, 578160/1663620)bytes
at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368) 
~[main/:na]
at 
org.apache.cassandra.index.internal.CassandraIndex.buildBlocking(CassandraIndex.java:688)
 ~[main/:na]
at 
org.apache.cassandra.index.internal.CassandraIndex.lambda$getBuildIndexTask$206(CassandraIndex.java:658)
 ~[main/:na]
at 
org.apache.cassandra.index.internal.CassandraIndex$$Lambda$151/1841229245.call(Unknown
 Source) ~[na:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_51]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_51]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] Caused by: 
java.util.concurrent.ExecutionException: 
org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
interrupted: Secondary index 
build@41202370-7c3e-11e5-9331-6bb6e58f8b1b(upgrade, cf, 
578160/{code}1663620)bytes
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_51]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_51]
at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:364) 
~[main/:na]
... 7 common frames omitted Caused by: 
org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
interrupted: Secondary index 
build@41202370-7c3e-11e5-9331-6bb6e58f8b1b(upgrade, cf, 578160/1663620)bytes
at 
org.apache.cassandra.index.SecondaryIndexBuilder.build(SecondaryIndexBuilder.java:67)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1269)
 ~[main/:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_51]
... 4 common frames omitted', 'ERROR [HintsDispatcher:2] 2015-10-27 
00:08:48,520 CassandraDaemon.java:195 - Exception in thread 
Thread[HintsDispatcher:2,1,main]', 'ERROR [HintsDispatcher:2] 2015-10-27 
00:11:58,336 CassandraDaemon.java:195 - Exception in thread 
Thread[HintsDispatcher:2,1,main]']
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10276) Do STCS in DTCS-windows

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977943#comment-14977943
 ] 

Marcus Eriksson commented on CASSANDRA-10276:
-

as we can tune that with STCS (ie lower min_threshold or change the 
bucket_low/high parameters), I think we are good.

> Do STCS in DTCS-windows
> ---
>
> Key: CASSANDRA-10276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10276
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.2.4, 2.1.12, 3.0.0
>
>
> To avoid constant recompaction of files in big ( > max threshold) DTCS 
> windows, we should do STCS of those files.
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10606) AbstractBTreePartition.rowCount() return the wrong number of rows for compact tables

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977959#comment-14977959
 ] 

Sylvain Lebresne commented on CASSANDRA-10606:
--

I don't think that's what we should do. We want the internal storage engine not 
to care about compact tables are much as possible: the use of statics for 
compact tables should be and stay a higher level concern and as far as 
{{AbstractBTreePartition.rowCount()}} is concerned, counting the static row 
_only_ if there is no other rows is imo a weird contract. I'll note in 
particular that this method is used in a number of place for some data 
structure sizing and this patch would actually break those places.

Why do you think this is a problem btw? (I'll note that all counting that is 
CQL related uses {{DataLimits}} which takes that kind of subtlety into account) 



> AbstractBTreePartition.rowCount() return the wrong number of rows for compact 
> tables
> 
>
> Key: CASSANDRA-10606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10606
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.0.0
>
> Attachments: 10606-3.0.txt
>
>
> For compact tables {{AbstractBTreePartition.rowCount()}} return the wrong 
> number of columns as it does not take into account static rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977863#comment-14977863
 ] 

Jeff Jirsa edited comment on CASSANDRA-10276 at 10/28/15 6:51 AM:
--

I'm definitely not disagreeing with you, but like always, the devil's in the 
details. If there were 3 40MB files, 3 120M files, 3 500M files, 3 2G files, 3 
8G files, and 3 24G files all in one window, it might be nice to do one big 
rollup compaction (and that's a pretty extreme example, but let's look at the 
edge cases, because someone's bound to hit it - one single 40M flush away from 
having a single 100G file in normal STCS operation, but in an old window, that 
next 40M file may never show up, so we'll end up keeping 17 extra sstables 
around for no reason).  


was (Author: jjirsa):
I'm definitely not disagreeing with you, but like always, the devil's in the 
details. If there were 3 40MB files, 3 120M files, 3 500M files, 3 2G files, 3 
8G files, and 3 24G files all in one window, it might be nice to do one big 
rollup compaction (and that's a pretty extreme example, but let's look at the 
edge cases, because someone's bound to hit it - one single 40M flush away from 
having a single 100G file in normal STCS operation).  

> With DTCS, do STCS in windows if more than max_threshold sstables
> -
>
> Key: CASSANDRA-10276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10276
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.x, 2.1.x, 2.2.x
>
>
> To avoid constant recompaction of files in big ( > max threshold) DTCS 
> windows, we should do STCS of those files.
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977863#comment-14977863
 ] 

Jeff Jirsa commented on CASSANDRA-10276:


I'm definitely not disagreeing with you, but like always, the devil's in the 
details. If there were 3 40MB files, 3 120M files, 3 500M files, 3 2G files, 3 
8G files, and 3 24G files all in one window, it might be nice to do one big 
rollup compaction (and that's a pretty extreme example, but let's look at the 
edge cases, because someone's bound to hit it - one single 40M flush away from 
having a single 100G file in normal STCS operation).  

> With DTCS, do STCS in windows if more than max_threshold sstables
> -
>
> Key: CASSANDRA-10276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10276
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.x, 2.1.x, 2.2.x
>
>
> To avoid constant recompaction of files in big ( > max threshold) DTCS 
> windows, we should do STCS of those files.
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Do size tiered compaction in date tiered compaction windows

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 78810f255 -> cedcf07c5


Do size tiered compaction in date tiered compaction windows

Patch by marcuse; reviewed by Jeff Jirsa for CASSANDRA-10276


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cedcf07c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cedcf07c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cedcf07c

Branch: refs/heads/cassandra-2.1
Commit: cedcf07c542235815c023b66f151ad8c7aa9ba9a
Parents: 78810f2
Author: Marcus Eriksson 
Authored: Mon Sep 7 10:39:15 2015 +0200
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:40:12 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2ca3b43..5b46eac 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
  * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index a8e2aff..ece596f 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -18,7 +18,6 @@
 package org.apache.cassandra.db.compaction;
 
 import java.util.*;
-import java.util.concurrent.TimeUnit;
 
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
@@ -40,6 +39,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 protected volatile int estimatedRemainingTasks;
 private final Set sstables = new HashSet<>();
 private long lastExpiredCheck;
+private final SizeTieredCompactionStrategyOptions stcsOptions;
 
 public DateTieredCompactionStrategy(ColumnFamilyStore cfs, Map options)
 {
@@ -54,6 +54,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 else
 logger.debug("Enabling tombstone compactions for DTCS");
 
+this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
 }
 
 @Override
@@ -137,7 +138,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

cfs.getMinimumCompactionThreshold(),

cfs.getMaximumCompactionThreshold(),
now,
-   options.baseTime);
+   options.baseTime,
+   stcsOptions);
 if (!mostInteresting.isEmpty())
 return mostInteresting;
 return null;
@@ -328,7 +330,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 for (List bucket : tasks)
 {
 if (bucket.size() >= cfs.getMinimumCompactionThreshold())
-n += Math.ceil((double)bucket.size() / 
cfs.getMaximumCompactionThreshold());
+n += getSTCSBuckets(bucket, stcsOptions).size();
 }
 estimatedRemainingTasks = n;
 }
@@ -341,7 +343,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @return a bucket (list) of sstables to compact.
  */
 @VisibleForTesting
-static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime)
+static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime, 
SizeTieredCompactionStrategyOptions stcsOptions)
 {
 // If the "incoming window" has at least minThreshold SSTables, choose 
that one.
 // For any other bucket, at least 2 SSTables is enough.
@@ -349,23 +351,31 @@ 

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a356a70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a356a70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a356a70

Branch: refs/heads/cassandra-2.2
Commit: 5a356a70631d50cbab0f01dbfea868f26af66582
Parents: 40cef77 cedcf07
Author: Marcus Eriksson 
Authored: Wed Oct 28 08:43:10 2015 +0100
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:50:44 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/CHANGES.txt
--
diff --cc CHANGES.txt
index 24862b0,5b46eac..9021e9d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 -2.1.12
 +2.2.4
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Do STCS in DTCS windows (CASSANDRA-10276)
   * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --cc 
src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index da2d35d,ece596f..fe8f135
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@@ -50,11 -49,12 +50,12 @@@ public class DateTieredCompactionStrate
  if 
(!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
 && !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
  {
  disableTombstoneCompactions = true;
 -logger.debug("Disabling tombstone compactions for DTCS");
 +logger.trace("Disabling tombstone compactions for DTCS");
  }
  else
 -logger.debug("Enabling tombstone compactions for DTCS");
 +logger.trace("Enabling tombstone compactions for DTCS");
  
+ this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
  }
  
  @Override
@@@ -372,14 -379,13 +382,14 @@@
  }
  
  @Override
 -public synchronized Collection getMaximalTask(int 
gcBefore)
 +@SuppressWarnings("resource")
 +public synchronized Collection getMaximalTask(int 
gcBefore, boolean splitOutput)
  {
 -Iterable sstables = cfs.markAllCompacting();
 -if (sstables == null)
 +LifecycleTransaction modifier = 
cfs.markAllCompacting(OperationType.COMPACTION);
 +if (modifier == null)
  return null;
  
- return Arrays.asList(new CompactionTask(cfs, 
modifier, gcBefore, false));
 -return Collections.singleton(new 
CompactionTask(cfs, sstables, gcBefore, false));
++return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore, false));
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index 0159c83,368101d..2824d3a
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@@ -325,10 -301,50 +312,50 @@@ public class DateTieredCompactionStrate
  Thread.sleep(2000);
  AbstractCompactionTask t = dtcs.getNextBackgroundTask((int) 
(System.currentTimeMillis()/1000));
  assertNotNull(t);
 -assertEquals(1, Iterables.size(t.sstables));
 -SSTableReader sstable = t.sstables.iterator().next();
 +assertEquals(1, Iterables.size(t.transaction.originals()));
 +SSTableReader sstable = t.transaction.originals().iterator().next();
  assertEquals(sstable, expiredSSTable);
 -

[4/4] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[marcuse]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5995de3b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5995de3b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5995de3b

Branch: refs/heads/trunk
Commit: 5995de3b638fdd898ddfb94c3e8d2bf3ed3cf657
Parents: 7cedeca 27ca491
Author: Marcus Eriksson 
Authored: Wed Oct 28 09:06:01 2015 +0100
Committer: Marcus Eriksson 
Committed: Wed Oct 28 09:06:01 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 +++-
 .../DateTieredCompactionStrategyTest.java   | 73 ++--
 3 files changed, 82 insertions(+), 38 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5995de3b/CHANGES.txt
--

[1/2] cassandra git commit: Do size tiered compaction in date tiered compaction windows

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 40cef770d -> 5a356a706


Do size tiered compaction in date tiered compaction windows

Patch by marcuse; reviewed by Jeff Jirsa for CASSANDRA-10276


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cedcf07c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cedcf07c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cedcf07c

Branch: refs/heads/cassandra-2.2
Commit: cedcf07c542235815c023b66f151ad8c7aa9ba9a
Parents: 78810f2
Author: Marcus Eriksson 
Authored: Mon Sep 7 10:39:15 2015 +0200
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:40:12 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2ca3b43..5b46eac 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
  * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index a8e2aff..ece596f 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -18,7 +18,6 @@
 package org.apache.cassandra.db.compaction;
 
 import java.util.*;
-import java.util.concurrent.TimeUnit;
 
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
@@ -40,6 +39,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 protected volatile int estimatedRemainingTasks;
 private final Set sstables = new HashSet<>();
 private long lastExpiredCheck;
+private final SizeTieredCompactionStrategyOptions stcsOptions;
 
 public DateTieredCompactionStrategy(ColumnFamilyStore cfs, Map options)
 {
@@ -54,6 +54,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 else
 logger.debug("Enabling tombstone compactions for DTCS");
 
+this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
 }
 
 @Override
@@ -137,7 +138,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

cfs.getMinimumCompactionThreshold(),

cfs.getMaximumCompactionThreshold(),
now,
-   options.baseTime);
+   options.baseTime,
+   stcsOptions);
 if (!mostInteresting.isEmpty())
 return mostInteresting;
 return null;
@@ -328,7 +330,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 for (List bucket : tasks)
 {
 if (bucket.size() >= cfs.getMinimumCompactionThreshold())
-n += Math.ceil((double)bucket.size() / 
cfs.getMaximumCompactionThreshold());
+n += getSTCSBuckets(bucket, stcsOptions).size();
 }
 estimatedRemainingTasks = n;
 }
@@ -341,7 +343,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @return a bucket (list) of sstables to compact.
  */
 @VisibleForTesting
-static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime)
+static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime, 
SizeTieredCompactionStrategyOptions stcsOptions)
 {
 // If the "incoming window" has at least minThreshold SSTables, choose 
that one.
 // For any other bucket, at least 2 SSTables is enough.
@@ -349,23 +351,31 @@ 

[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a356a70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a356a70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a356a70

Branch: refs/heads/cassandra-3.0
Commit: 5a356a70631d50cbab0f01dbfea868f26af66582
Parents: 40cef77 cedcf07
Author: Marcus Eriksson 
Authored: Wed Oct 28 08:43:10 2015 +0100
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:50:44 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/CHANGES.txt
--
diff --cc CHANGES.txt
index 24862b0,5b46eac..9021e9d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 -2.1.12
 +2.2.4
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Do STCS in DTCS windows (CASSANDRA-10276)
   * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --cc 
src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index da2d35d,ece596f..fe8f135
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@@ -50,11 -49,12 +50,12 @@@ public class DateTieredCompactionStrate
  if 
(!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
 && !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
  {
  disableTombstoneCompactions = true;
 -logger.debug("Disabling tombstone compactions for DTCS");
 +logger.trace("Disabling tombstone compactions for DTCS");
  }
  else
 -logger.debug("Enabling tombstone compactions for DTCS");
 +logger.trace("Enabling tombstone compactions for DTCS");
  
+ this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
  }
  
  @Override
@@@ -372,14 -379,13 +382,14 @@@
  }
  
  @Override
 -public synchronized Collection getMaximalTask(int 
gcBefore)
 +@SuppressWarnings("resource")
 +public synchronized Collection getMaximalTask(int 
gcBefore, boolean splitOutput)
  {
 -Iterable sstables = cfs.markAllCompacting();
 -if (sstables == null)
 +LifecycleTransaction modifier = 
cfs.markAllCompacting(OperationType.COMPACTION);
 +if (modifier == null)
  return null;
  
- return Arrays.asList(new CompactionTask(cfs, 
modifier, gcBefore, false));
 -return Collections.singleton(new 
CompactionTask(cfs, sstables, gcBefore, false));
++return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore, false));
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index 0159c83,368101d..2824d3a
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@@ -325,10 -301,50 +312,50 @@@ public class DateTieredCompactionStrate
  Thread.sleep(2000);
  AbstractCompactionTask t = dtcs.getNextBackgroundTask((int) 
(System.currentTimeMillis()/1000));
  assertNotNull(t);
 -assertEquals(1, Iterables.size(t.sstables));
 -SSTableReader sstable = t.sstables.iterator().next();
 +assertEquals(1, Iterables.size(t.transaction.originals()));
 +SSTableReader sstable = t.transaction.originals().iterator().next();
  assertEquals(sstable, expiredSSTable);
 -

[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27ca4915
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27ca4915
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27ca4915

Branch: refs/heads/cassandra-3.0
Commit: 27ca4915e9a08c9582ebf077f2cc66821494f0b9
Parents: d856d3d 5a356a7
Author: Marcus Eriksson 
Authored: Wed Oct 28 09:04:30 2015 +0100
Committer: Marcus Eriksson 
Committed: Wed Oct 28 09:04:30 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 +++-
 .../DateTieredCompactionStrategyTest.java   | 73 ++--
 3 files changed, 82 insertions(+), 38 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27ca4915/CHANGES.txt
--
diff --cc CHANGES.txt
index f8dffb0,9021e9d..9b81b6e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,12 +1,18 @@@
 -2.2.4
 +3.0
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Do STCS in DTCS windows (CASSANDRA-10276)
 - * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)
   * Fix paging issues with partitions containing only static columns data 
(CASSANDRA-10381)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27ca4915/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --cc 
src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index cfb0079,fe8f135..65fec2b
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@@ -380,7 -389,7 +390,7 @@@ public class DateTieredCompactionStrate
  if (modifier == null)
  return null;
  
- return Arrays.asList(new CompactionTask(cfs, 
modifier, gcBefore));
 -return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore, false));
++return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore));
  }
  
  @Override
@@@ -428,9 -437,11 +438,11 @@@
  Map uncheckedOptions = 
AbstractCompactionStrategy.validateOptions(options);
  uncheckedOptions = 
DateTieredCompactionStrategyOptions.validateOptions(options, uncheckedOptions);
  
 -uncheckedOptions.remove(CFPropDefs.KW_MINCOMPACTIONTHRESHOLD);
 -uncheckedOptions.remove(CFPropDefs.KW_MAXCOMPACTIONTHRESHOLD);
 +
uncheckedOptions.remove(CompactionParams.Option.MIN_THRESHOLD.toString());
 +
uncheckedOptions.remove(CompactionParams.Option.MAX_THRESHOLD.toString());
  
+ uncheckedOptions = 
SizeTieredCompactionStrategyOptions.validateOptions(options, uncheckedOptions);
+ 
  return uncheckedOptions;
  }
  

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27ca4915/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index 003a1cf,2824d3a..01a6dfa
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@@ -32,10 -32,11 +32,11 @@@ import org.apache.cassandra.Util
  import org.apache.cassandra.db.ColumnFamilyStore;
  import 

[2/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a356a70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a356a70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a356a70

Branch: refs/heads/trunk
Commit: 5a356a70631d50cbab0f01dbfea868f26af66582
Parents: 40cef77 cedcf07
Author: Marcus Eriksson 
Authored: Wed Oct 28 08:43:10 2015 +0100
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:50:44 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/CHANGES.txt
--
diff --cc CHANGES.txt
index 24862b0,5b46eac..9021e9d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 -2.1.12
 +2.2.4
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Do STCS in DTCS windows (CASSANDRA-10276)
   * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --cc 
src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index da2d35d,ece596f..fe8f135
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@@ -50,11 -49,12 +50,12 @@@ public class DateTieredCompactionStrate
  if 
(!options.containsKey(AbstractCompactionStrategy.TOMBSTONE_COMPACTION_INTERVAL_OPTION)
 && !options.containsKey(AbstractCompactionStrategy.TOMBSTONE_THRESHOLD_OPTION))
  {
  disableTombstoneCompactions = true;
 -logger.debug("Disabling tombstone compactions for DTCS");
 +logger.trace("Disabling tombstone compactions for DTCS");
  }
  else
 -logger.debug("Enabling tombstone compactions for DTCS");
 +logger.trace("Enabling tombstone compactions for DTCS");
  
+ this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
  }
  
  @Override
@@@ -372,14 -379,13 +382,14 @@@
  }
  
  @Override
 -public synchronized Collection getMaximalTask(int 
gcBefore)
 +@SuppressWarnings("resource")
 +public synchronized Collection getMaximalTask(int 
gcBefore, boolean splitOutput)
  {
 -Iterable sstables = cfs.markAllCompacting();
 -if (sstables == null)
 +LifecycleTransaction modifier = 
cfs.markAllCompacting(OperationType.COMPACTION);
 +if (modifier == null)
  return null;
  
- return Arrays.asList(new CompactionTask(cfs, 
modifier, gcBefore, false));
 -return Collections.singleton(new 
CompactionTask(cfs, sstables, gcBefore, false));
++return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore, false));
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a356a70/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index 0159c83,368101d..2824d3a
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@@ -325,10 -301,50 +312,50 @@@ public class DateTieredCompactionStrate
  Thread.sleep(2000);
  AbstractCompactionTask t = dtcs.getNextBackgroundTask((int) 
(System.currentTimeMillis()/1000));
  assertNotNull(t);
 -assertEquals(1, Iterables.size(t.sstables));
 -SSTableReader sstable = t.sstables.iterator().next();
 +assertEquals(1, Iterables.size(t.transaction.originals()));
 +SSTableReader sstable = t.transaction.originals().iterator().next();
  assertEquals(sstable, expiredSSTable);
 -

[1/3] cassandra git commit: Do size tiered compaction in date tiered compaction windows

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 d856d3d7b -> 27ca4915e


Do size tiered compaction in date tiered compaction windows

Patch by marcuse; reviewed by Jeff Jirsa for CASSANDRA-10276


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cedcf07c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cedcf07c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cedcf07c

Branch: refs/heads/cassandra-3.0
Commit: cedcf07c542235815c023b66f151ad8c7aa9ba9a
Parents: 78810f2
Author: Marcus Eriksson 
Authored: Mon Sep 7 10:39:15 2015 +0200
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:40:12 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2ca3b43..5b46eac 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
  * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index a8e2aff..ece596f 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -18,7 +18,6 @@
 package org.apache.cassandra.db.compaction;
 
 import java.util.*;
-import java.util.concurrent.TimeUnit;
 
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
@@ -40,6 +39,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 protected volatile int estimatedRemainingTasks;
 private final Set sstables = new HashSet<>();
 private long lastExpiredCheck;
+private final SizeTieredCompactionStrategyOptions stcsOptions;
 
 public DateTieredCompactionStrategy(ColumnFamilyStore cfs, Map options)
 {
@@ -54,6 +54,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 else
 logger.debug("Enabling tombstone compactions for DTCS");
 
+this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
 }
 
 @Override
@@ -137,7 +138,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

cfs.getMinimumCompactionThreshold(),

cfs.getMaximumCompactionThreshold(),
now,
-   options.baseTime);
+   options.baseTime,
+   stcsOptions);
 if (!mostInteresting.isEmpty())
 return mostInteresting;
 return null;
@@ -328,7 +330,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 for (List bucket : tasks)
 {
 if (bucket.size() >= cfs.getMinimumCompactionThreshold())
-n += Math.ceil((double)bucket.size() / 
cfs.getMaximumCompactionThreshold());
+n += getSTCSBuckets(bucket, stcsOptions).size();
 }
 estimatedRemainingTasks = n;
 }
@@ -341,7 +343,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @return a bucket (list) of sstables to compact.
  */
 @VisibleForTesting
-static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime)
+static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime, 
SizeTieredCompactionStrategyOptions stcsOptions)
 {
 // If the "incoming window" has at least minThreshold SSTables, choose 
that one.
 // For any other bucket, at least 2 SSTables is enough.
@@ -349,23 +351,31 @@ 

[3/4] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27ca4915
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27ca4915
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27ca4915

Branch: refs/heads/trunk
Commit: 27ca4915e9a08c9582ebf077f2cc66821494f0b9
Parents: d856d3d 5a356a7
Author: Marcus Eriksson 
Authored: Wed Oct 28 09:04:30 2015 +0100
Committer: Marcus Eriksson 
Committed: Wed Oct 28 09:04:30 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 +++-
 .../DateTieredCompactionStrategyTest.java   | 73 ++--
 3 files changed, 82 insertions(+), 38 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27ca4915/CHANGES.txt
--
diff --cc CHANGES.txt
index f8dffb0,9021e9d..9b81b6e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,12 +1,18 @@@
 -2.2.4
 +3.0
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Support encrypted and plain traffic on the same port (CASSANDRA-10559)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Do STCS in DTCS windows (CASSANDRA-10276)
 - * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
   * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
   * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)
   * Fix paging issues with partitions containing only static columns data 
(CASSANDRA-10381)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27ca4915/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --cc 
src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index cfb0079,fe8f135..65fec2b
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@@ -380,7 -389,7 +390,7 @@@ public class DateTieredCompactionStrate
  if (modifier == null)
  return null;
  
- return Arrays.asList(new CompactionTask(cfs, 
modifier, gcBefore));
 -return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore, false));
++return Collections.singleton(new 
CompactionTask(cfs, modifier, gcBefore));
  }
  
  @Override
@@@ -428,9 -437,11 +438,11 @@@
  Map uncheckedOptions = 
AbstractCompactionStrategy.validateOptions(options);
  uncheckedOptions = 
DateTieredCompactionStrategyOptions.validateOptions(options, uncheckedOptions);
  
 -uncheckedOptions.remove(CFPropDefs.KW_MINCOMPACTIONTHRESHOLD);
 -uncheckedOptions.remove(CFPropDefs.KW_MAXCOMPACTIONTHRESHOLD);
 +
uncheckedOptions.remove(CompactionParams.Option.MIN_THRESHOLD.toString());
 +
uncheckedOptions.remove(CompactionParams.Option.MAX_THRESHOLD.toString());
  
+ uncheckedOptions = 
SizeTieredCompactionStrategyOptions.validateOptions(options, uncheckedOptions);
+ 
  return uncheckedOptions;
  }
  

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27ca4915/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index 003a1cf,2824d3a..01a6dfa
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@@ -32,10 -32,11 +32,11 @@@ import org.apache.cassandra.Util
  import org.apache.cassandra.db.ColumnFamilyStore;
  import org.apache.cassandra.db.DecoratedKey;
  

[jira] [Updated] (CASSANDRA-10276) Do STCS in DTCS-windows

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-10276:

Summary: Do STCS in DTCS-windows  (was: With DTCS, do STCS in windows if 
more than max_threshold sstables)

> Do STCS in DTCS-windows
> ---
>
> Key: CASSANDRA-10276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10276
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.x, 2.1.x, 2.2.x
>
>
> To avoid constant recompaction of files in big ( > max threshold) DTCS 
> windows, we should do STCS of those files.
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/4] cassandra git commit: Do size tiered compaction in date tiered compaction windows

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7cedeca9a -> 5995de3b6


Do size tiered compaction in date tiered compaction windows

Patch by marcuse; reviewed by Jeff Jirsa for CASSANDRA-10276


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cedcf07c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cedcf07c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cedcf07c

Branch: refs/heads/trunk
Commit: cedcf07c542235815c023b66f151ad8c7aa9ba9a
Parents: 78810f2
Author: Marcus Eriksson 
Authored: Mon Sep 7 10:39:15 2015 +0200
Committer: Marcus Eriksson 
Committed: Wed Oct 28 08:40:12 2015 +0100

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 46 ++--
 .../DateTieredCompactionStrategyTest.java   | 57 ++--
 3 files changed, 72 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2ca3b43..5b46eac 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Do STCS in DTCS windows (CASSANDRA-10276)
  * Don't try to get ancestors from half-renamed sstables (CASSANDRA-10501)
  * Avoid repetition of JVM_OPTS in debian package (CASSANDRA-10251)
  * Fix potential NPE from handling result of SIM.highestSelectivityIndex 
(CASSANDRA-10550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cedcf07c/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index a8e2aff..ece596f 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -18,7 +18,6 @@
 package org.apache.cassandra.db.compaction;
 
 import java.util.*;
-import java.util.concurrent.TimeUnit;
 
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
@@ -40,6 +39,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 protected volatile int estimatedRemainingTasks;
 private final Set sstables = new HashSet<>();
 private long lastExpiredCheck;
+private final SizeTieredCompactionStrategyOptions stcsOptions;
 
 public DateTieredCompactionStrategy(ColumnFamilyStore cfs, Map options)
 {
@@ -54,6 +54,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 else
 logger.debug("Enabling tombstone compactions for DTCS");
 
+this.stcsOptions = new SizeTieredCompactionStrategyOptions(options);
 }
 
 @Override
@@ -137,7 +138,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

cfs.getMinimumCompactionThreshold(),

cfs.getMaximumCompactionThreshold(),
now,
-   options.baseTime);
+   options.baseTime,
+   stcsOptions);
 if (!mostInteresting.isEmpty())
 return mostInteresting;
 return null;
@@ -328,7 +330,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 for (List bucket : tasks)
 {
 if (bucket.size() >= cfs.getMinimumCompactionThreshold())
-n += Math.ceil((double)bucket.size() / 
cfs.getMaximumCompactionThreshold());
+n += getSTCSBuckets(bucket, stcsOptions).size();
 }
 estimatedRemainingTasks = n;
 }
@@ -341,7 +343,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @return a bucket (list) of sstables to compact.
  */
 @VisibleForTesting
-static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime)
+static List newestBucket(List buckets, 
int minThreshold, int maxThreshold, long now, long baseTime, 
SizeTieredCompactionStrategyOptions stcsOptions)
 {
 // If the "incoming window" has at least minThreshold SSTables, choose 
that one.
 // For any other bucket, at least 2 SSTables is enough.
@@ -349,23 +351,31 @@ public class 

[jira] [Commented] (CASSANDRA-10598) Unable to use same index name on column families which have different names in different keyspaces

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978086#comment-14978086
 ] 

ASF GitHub Bot commented on CASSANDRA-10598:


GitHub user allenyeh79 opened a pull request:

https://github.com/apache/cassandra/pull/56

CASSANDRA-10598:Let same index name can be used in different keyspaces

Let same index name can be used in different keyspaces.

JIRA link:
https://issues.apache.org/jira/browse/CASSANDRA-10598

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bitus-dt/cassandra cassandra-1.2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cassandra/pull/56.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #56


commit 85487ad55c253ac43d4980cf30b9cfeac256af0f
Author: allen.yeh 
Date:   2015-10-28T09:36:22Z

CASSANDRA-10598:The same index name can be used in different keyspaces
without conflict




> Unable to use same index name on column families which have different names 
> in different keyspaces
> --
>
> Key: CASSANDRA-10598
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10598
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Yeh Sheng Hsiung
>
> Expected:
> The same index name can be used in different keyspaces without conflict
> Actual:
> Unable to use same index name on column families which have different names 
> in different keyspaces
> Steps to Reproduce:
> {code}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 
> 'NetworkTopologyStrategy', 'DC1' : 1};
> cqlsh> CREATE KEYSPACE ks2 WITH replication = {'class': 
> 'NetworkTopologyStrategy', 'DC1' : 1};
> cqlsh> use ks1;
> cqlsh:ks1> CREATE TABLE t1 (c1 int PRIMARY KEY, c2 int);
> cqlsh:ks1> create index c2_idx on t1 (c2);
> cqlsh:ks1> use ks2;
> cqlsh:ks2> CREATE TABLE t2 (c1 int PRIMARY KEY, c2 int);
> cqlsh:ks2> create index c2_idx on t2 (c2);
> ConfigurationException:  configuration issue] message="Duplicate index name c2_idx">
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978123#comment-14978123
 ] 

Branimir Lambov commented on CASSANDRA-10515:
-

Let us know if that fix does not solve the problem. On the first form, did 
re-levelling improve the situation?

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: commitlog, triage
> Attachments: C5commitLogIncrease.jpg, CommitLogProblem.jpg, 
> CommitLogSize.jpg, MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, 
> cassandra.yaml, cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10606) PartitionUpdate.operationCount() return the wrong number of operation for compact tables

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10606:
---
Attachment: 10606-3.0-V2.txt

The patch count an extra operation in {{PartitionUpdate.operationCount()}} when 
the static row is modified and add unit tests to verify the behaviour.

CI:
* [unit tests 
results|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10606-3.0-testall/2/]
* [dtests 
results|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10606-3.0-dtest/3/]

> PartitionUpdate.operationCount() return the wrong number of operation for 
> compact tables
> 
>
> Key: CASSANDRA-10606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10606
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.0.0
>
> Attachments: 10606-3.0-V2.txt, 10606-3.0.txt
>
>
> For compact tables {{PartitionUpdate.operationCount()}} return the wrong 
> number of operation as it does not take into operation on static rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Fix PartitionUpdate.operationCount()for updates with static column

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9be01e34c -> a4fe865d4


Fix PartitionUpdate.operationCount()for updates with static column

patch by Benjamin Lerer; reviewed by Sylvain Lebresne for CASSANDRA-10606


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3615da58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3615da58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3615da58

Branch: refs/heads/trunk
Commit: 3615da58310a778c7ee6d9b2d77fd3fb7a6700f4
Parents: 4d00468
Author: Benjamin Lerer 
Authored: Wed Oct 28 17:41:03 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Oct 28 17:41:03 2015 +0100

--
 CHANGES.txt |  1 +
 .../db/partitions/PartitionUpdate.java  |  1 +
 .../db/partition/PartitionUpdateTest.java   | 67 
 3 files changed, 69 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3615da58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3423a1e..c51cb51 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
  * Fix thrift get() queries with defined columns (CASSANDRA-10586)
  * Fix marking of indexes as built and removed (CASSANDRA-10601)
  * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3615da58/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
--
diff --git a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java 
b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
index b1776ca..52f8f67 100644
--- a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
+++ b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
@@ -334,6 +334,7 @@ public class PartitionUpdate extends AbstractBTreePartition
 public int operationCount()
 {
 return rowCount()
+ + (staticRow().isEmpty() ? 0 : 1)
  + deletionInfo.rangeCount()
  + (deletionInfo.getPartitionDeletion().isLive() ? 0 : 1);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3615da58/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java 
b/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java
new file mode 100644
index 000..a069db1
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.partition;
+
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.db.RowUpdateBuilder;
+import org.apache.cassandra.db.partitions.PartitionUpdate;
+import org.apache.cassandra.utils.FBUtilities;
+import org.junit.Test;
+
+import junit.framework.Assert;
+
+public class PartitionUpdateTest extends CQLTester
+{
+@Test
+public void testOperationCount()
+{
+createTable("CREATE TABLE %s (key text, clustering int, a int, s int 
static, PRIMARY KEY(key, clustering))");
+CFMetaData cfm = currentTableMetadata();
+
+long timestamp = FBUtilities.timestampMicros();
+PartitionUpdate update = new RowUpdateBuilder(cfm, timestamp, 
"key0").clustering(1).add("a", 1).buildUpdate();
+Assert.assertEquals(1, update.operationCount());
+
+update = new RowUpdateBuilder(cfm, timestamp, "key0").buildUpdate();
+Assert.assertEquals(0, update.operationCount());
+
+update = new RowUpdateBuilder(cfm, timestamp, "key0").add("s", 

[jira] [Updated] (CASSANDRA-10614) AssertionError while flushing memtables

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-10614:

Summary: AssertionError while flushing memtables  (was: AsertionError while 
flushing memtables)

> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: T Jake Luciani
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10581) Update cassandra.yaml comments to reflect memory_allocator deprecation, remove in 3.0

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978730#comment-14978730
 ] 

Jim Witschey commented on CASSANDRA-10581:
--

The documentation changes look good to me. If it's not already logged somewhere 
that I missed it, could you add some kind of logging indicating when Cassandra 
used JEMAlloc? As is, it's difficult to determine if it's actually used, as far 
as I can tell.

> Update cassandra.yaml comments to reflect memory_allocator deprecation, 
> remove in 3.0
> -
>
> Key: CASSANDRA-10581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10581
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Robert Stupp
> Fix For: 2.2.x, 3.0.0
>
>
> Looks like in 2.2+ changing the {{memory_allocator}} field in cassandra.yaml 
> has no effect:
> https://github.com/apache/cassandra/commit/0d2ec11c7e0abfb84d872289af6d3ac386cf381f#diff-b66584c9ce7b64019b5db5a531deeda1R207
> The instructions in comments on how to use jemalloc haven't been updated to 
> reflect this change. [~snazy], is that an accurate assessment?
> [~iamaleksey] How do we want to handle the deprecation more generally? Warn 
> on 2.2, remove in 3.0?
> EDIT: I described this in a way that isn't very clear to those unfamiliar 
> with Robert's change. To be clear: You can still use JEMAlloc with Cassandra. 
> The {{memory_allocator}} option was deprecated in favor of automatically 
> detecting if JEMAlloc is installed, and, if so, using it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10606) PartitionUpdate.operationCount() return the wrong number of operation for compact tables

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978739#comment-14978739
 ] 

Benjamin Lerer commented on CASSANDRA-10606:


Committed in 3.0 at 3615da58310a778c7ee6d9b2d77fd3fb7a6700f4 and merged into 
trunk.

> PartitionUpdate.operationCount() return the wrong number of operation for 
> compact tables
> 
>
> Key: CASSANDRA-10606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10606
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.0.0
>
> Attachments: 10606-3.0-V2.txt, 10606-3.0-V3.txt, 10606-3.0.txt
>
>
> For compact tables {{PartitionUpdate.operationCount()}} return the wrong 
> number of operation as it does not take into operation on static rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10581) Update cassandra.yaml comments to reflect memory_allocator deprecation, remove in 3.0

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10581:
-
Description: 
Looks like in 2.2+ changing the {{memory_allocator}} field in cassandra.yaml 
has no effect:

https://github.com/apache/cassandra/commit/0d2ec11c7e0abfb84d872289af6d3ac386cf381f#diff-b66584c9ce7b64019b5db5a531deeda1R207

The instructions in comments on how to use jemalloc haven't been updated to 
reflect this change. [~snazy], is that an accurate assessment?

[~iamaleksey] How do we want to handle the deprecation more generally? Warn on 
2.2, remove in 3.0?

EDIT: I described this in a way that isn't very clear to those unfamiliar with 
Robert's change. To be clear: You can still use JEMAlloc with Cassandra. The 
{{memory_allocator}} option was deprecated in favor of automatically detecting 
if JEMAlloc is installed, and, if so, using it. 

  was:
Looks like in 2.2+ changing the {{memory_allocator}} field in cassandra.yaml 
has no effect:

https://github.com/apache/cassandra/commit/0d2ec11c7e0abfb84d872289af6d3ac386cf381f#diff-b66584c9ce7b64019b5db5a531deeda1R207

The instructions in comments on how to use jemalloc haven't been updated to 
reflect this change. [~snazy], is that an accurate assessment?

[~iamaleksey] How do we want to handle the deprecation more generally? Warn on 
2.2, remove in 3.0?


> Update cassandra.yaml comments to reflect memory_allocator deprecation, 
> remove in 3.0
> -
>
> Key: CASSANDRA-10581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10581
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Robert Stupp
> Fix For: 2.2.x, 3.0.0
>
>
> Looks like in 2.2+ changing the {{memory_allocator}} field in cassandra.yaml 
> has no effect:
> https://github.com/apache/cassandra/commit/0d2ec11c7e0abfb84d872289af6d3ac386cf381f#diff-b66584c9ce7b64019b5db5a531deeda1R207
> The instructions in comments on how to use jemalloc haven't been updated to 
> reflect this change. [~snazy], is that an accurate assessment?
> [~iamaleksey] How do we want to handle the deprecation more generally? Warn 
> on 2.2, remove in 3.0?
> EDIT: I described this in a way that isn't very clear to those unfamiliar 
> with Robert's change. To be clear: You can still use JEMAlloc with Cassandra. 
> The {{memory_allocator}} option was deprecated in favor of automatically 
> detecting if JEMAlloc is installed, and, if so, using it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix PartitionUpdate.operationCount()for updates with static column

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 4d00468cc -> 3615da583


Fix PartitionUpdate.operationCount()for updates with static column

patch by Benjamin Lerer; reviewed by Sylvain Lebresne for CASSANDRA-10606


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3615da58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3615da58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3615da58

Branch: refs/heads/cassandra-3.0
Commit: 3615da58310a778c7ee6d9b2d77fd3fb7a6700f4
Parents: 4d00468
Author: Benjamin Lerer 
Authored: Wed Oct 28 17:41:03 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Oct 28 17:41:03 2015 +0100

--
 CHANGES.txt |  1 +
 .../db/partitions/PartitionUpdate.java  |  1 +
 .../db/partition/PartitionUpdateTest.java   | 67 
 3 files changed, 69 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3615da58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3423a1e..c51cb51 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
  * Fix thrift get() queries with defined columns (CASSANDRA-10586)
  * Fix marking of indexes as built and removed (CASSANDRA-10601)
  * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3615da58/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
--
diff --git a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java 
b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
index b1776ca..52f8f67 100644
--- a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
+++ b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
@@ -334,6 +334,7 @@ public class PartitionUpdate extends AbstractBTreePartition
 public int operationCount()
 {
 return rowCount()
+ + (staticRow().isEmpty() ? 0 : 1)
  + deletionInfo.rangeCount()
  + (deletionInfo.getPartitionDeletion().isLive() ? 0 : 1);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3615da58/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java 
b/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java
new file mode 100644
index 000..a069db1
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/partition/PartitionUpdateTest.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.partition;
+
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.db.RowUpdateBuilder;
+import org.apache.cassandra.db.partitions.PartitionUpdate;
+import org.apache.cassandra.utils.FBUtilities;
+import org.junit.Test;
+
+import junit.framework.Assert;
+
+public class PartitionUpdateTest extends CQLTester
+{
+@Test
+public void testOperationCount()
+{
+createTable("CREATE TABLE %s (key text, clustering int, a int, s int 
static, PRIMARY KEY(key, clustering))");
+CFMetaData cfm = currentTableMetadata();
+
+long timestamp = FBUtilities.timestampMicros();
+PartitionUpdate update = new RowUpdateBuilder(cfm, timestamp, 
"key0").clustering(1).add("a", 1).buildUpdate();
+Assert.assertEquals(1, update.operationCount());
+
+update = new RowUpdateBuilder(cfm, timestamp, "key0").buildUpdate();
+Assert.assertEquals(0, update.operationCount());
+
+update = new RowUpdateBuilder(cfm, timestamp, 

[jira] [Created] (CASSANDRA-10614) AsertionError while flushing memtables

[Tyler Hobbs (JIRA)]

Tyler Hobbs created CASSANDRA-10614:
---

 Summary: AsertionError while flushing memtables
 Key: CASSANDRA-10614
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Priority: Critical
 Fix For: 3.0.0


While running mvbench against a single local node, I noticed this stacktrace 
showing up multiple times in the logs:

{noformat}
ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
~[main/:na]
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
 ~[main/:na]
at 
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
~[main/:na]
at 
org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
 ~[main/:na]
at 
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
~[main/:na]
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
 ~[main/:na]
at 
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
 ~[main/:na]
at 
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
 ~[main/:na]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
 ~[main/:na]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
~[main/:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[main/:na]
at 
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
 ~[guava-18.0.jar:na]
at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
 ~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
{noformat}

To reproduce, run mvbench with the regular schema and the following arguments:

{noformat}
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists 
1 -n 50 --endpoint 127.0.0.1"
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10604) Secondary index metadata is not reloaded when table is altered

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978661#comment-14978661
 ] 

Tyler Hobbs commented on CASSANDRA-10604:
-

+1 on the code changes.  The testall failures either don't seem to be related 
or are not regressions.  Assuming the dtest results look okay, you should be 
good to commit.

> Secondary index metadata is not reloaded when table is altered
> --
>
> Key: CASSANDRA-10604
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10604
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> The javadocs for {{Index.getMetadataReloadTask()}} state the following:
> {quote}
> Returns a task to reload the internal metadata of an index.
> Called when the base table metadata is modified or when the configuration of 
> the Index is updated.
> {quote}
> However, altering a table does not result in the reload task being executed.  
> I think the root of the problem is that in 
> {{SecondaryIndexManager.reloadIndex()}}, we only execute the reload task when 
> the old {{IndexMetadata}} does not equal the current {{IndexMetadata}}.  
> Altering the table does not change the index metadata, so this check always 
> fails.
> This especially affects per-row secondary indexes, where the index may need 
> to handle columns being added or dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9829) Dynamically adjust LCS level sizes

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9829:
---
Assignee: (was: Paulo Motta)

> Dynamically adjust LCS level sizes
> --
>
> Key: CASSANDRA-9829
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9829
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>  Labels: compaction, performance
> Fix For: 3.x
>
>
> LCS works best when the top level is full.  Then 90% of reads can be served 
> from a single sstable.  By contrast if the top level is only 10% full then 
> 90% of reads will be served from two.  This results in worse performance as 
> well as confused users.
> To address this, we can adjust the ideal top level size to how much data is 
> actually in it (and set each corresponding lower level to 1/10 of the next 
> one above).
> (This is an idea [from 
> rocksdb|https://www.reddit.com/r/IAmA/comments/3de3cv/we_are_rocksdb_engineering_team_ask_us_anything/ct4asen].)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10614) AssertionError while flushing memtables

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani reassigned CASSANDRA-10614:
--

Assignee: T Jake Luciani

> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: T Jake Luciani
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8844) Change Data Capture (CDC)

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8844:
---
Component/s: Local Write-Read Paths
 Coordination

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.x
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order to make consuming a change log easy and efficient to do with low 
> latency, the following could supplement the approach outlined above
> - Instead of writing to a logfile, by default, Cassandra could expose a 
> socket for a daemon 

[jira] [Assigned] (CASSANDRA-10614) AsertionError while flushing memtables

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-10614:
---

Assignee: T Jake Luciani

> AsertionError while flushing memtables
> --
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: T Jake Luciani
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch cassandra-3.0 into trunk

[blerer]

Merge branch cassandra-3.0 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a4fe865d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a4fe865d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a4fe865d

Branch: refs/heads/trunk
Commit: a4fe865d4f8cb7ca8d14191d6d9ccf6ac1f79eb3
Parents: 9be01e3 3615da5
Author: Benjamin Lerer 
Authored: Wed Oct 28 17:47:07 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Oct 28 17:47:33 2015 +0100

--
 CHANGES.txt |  1 +
 .../db/partitions/PartitionUpdate.java  |  1 +
 .../db/partition/PartitionUpdateTest.java   | 67 
 3 files changed, 69 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a4fe865d/CHANGES.txt
--
diff --cc CHANGES.txt
index 04cefae,c51cb51..29117e0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
   * Fix thrift get() queries with defined columns (CASSANDRA-10586)
   * Fix marking of indexes as built and removed (CASSANDRA-10601)
   * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)

[jira] [Updated] (CASSANDRA-10614) AssertionError while flushing memtables

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-10614:
---
Assignee: (was: T Jake Luciani)

> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978622#comment-14978622
 ] 

Adam Holmberg commented on CASSANDRA-10365:
---

Repeating here for additional contributors.
Python dev branch for this is 
[422|https://github.com/datastax/python-driver/tree/422]

- Requires CASSANDRA-10513 patch for use with cqlsh
- Still testing functionality outside of metadata integration

> Consider storing types by their CQL names in schema tables instead of 
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10605) MessagingService: COUNTER_MUTATION verb is associated with MUTATION stage

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10605:
-
Priority: Major  (was: Minor)

> MessagingService: COUNTER_MUTATION verb is associated with MUTATION stage
> -
>
> Key: CASSANDRA-10605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10605
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Anubhav Kale
> Fix For: 2.1.10
>
> Attachments: trunk-10605.patch
>
>
> In MessagingService.java, I see that the verb COUNTER_MUTATION is associated 
> with COUNTER_MUTATION stage first, and later with MUTATION stage. We should 
> remove appropriate entry.
> I marked this as Major because I think the stage assignment is wrong.
> What should this be assigned to ?
> public static final EnumMap verbStages = new 
> EnumMap(MessagingService.Verb.class)
> {{
> put(Verb.MUTATION, Stage.MUTATION);
> put(Verb.COUNTER_MUTATION, Stage.COUNTER_MUTATION);*
> put(Verb.READ_REPAIR, Stage.MUTATION);
> put(Verb.HINT, Stage.MUTATION);
> put(Verb.TRUNCATE, Stage.MUTATION);
> put(Verb.PAXOS_PREPARE, Stage.MUTATION);
> put(Verb.PAXOS_PROPOSE, Stage.MUTATION);
> put(Verb.PAXOS_COMMIT, Stage.MUTATION);
> put(Verb.BATCH_STORE, Stage.MUTATION);
> put(Verb.BATCH_REMOVE, Stage.MUTATION);
> put(Verb.READ, Stage.READ);
> put(Verb.RANGE_SLICE, Stage.READ);
> put(Verb.INDEX_SCAN, Stage.READ);
> put(Verb.PAGED_RANGE, Stage.READ);
> put(Verb.REQUEST_RESPONSE, Stage.REQUEST_RESPONSE);
> put(Verb.INTERNAL_RESPONSE, Stage.INTERNAL_RESPONSE);
> put(Verb.STREAM_REPLY, Stage.MISC); // actually handled by 
> FileStreamTask and streamExecutors
> put(Verb.STREAM_REQUEST, Stage.MISC);
> put(Verb.REPLICATION_FINISHED, Stage.MISC);
> put(Verb.SNAPSHOT, Stage.MISC);
> put(Verb.TREE_REQUEST, Stage.ANTI_ENTROPY);
> put(Verb.TREE_RESPONSE, Stage.ANTI_ENTROPY);
> put(Verb.STREAMING_REPAIR_REQUEST, Stage.ANTI_ENTROPY);
> put(Verb.STREAMING_REPAIR_RESPONSE, Stage.ANTI_ENTROPY);
> put(Verb.REPAIR_MESSAGE, Stage.ANTI_ENTROPY);
> put(Verb.GOSSIP_DIGEST_ACK, Stage.GOSSIP);
> put(Verb.GOSSIP_DIGEST_ACK2, Stage.GOSSIP);
> put(Verb.GOSSIP_DIGEST_SYN, Stage.GOSSIP);
> put(Verb.GOSSIP_SHUTDOWN, Stage.GOSSIP);
> put(Verb.DEFINITIONS_UPDATE, Stage.MIGRATION);
> put(Verb.SCHEMA_CHECK, Stage.MIGRATION);
> put(Verb.MIGRATION_REQUEST, Stage.MIGRATION);
> put(Verb.INDEX_SCAN, Stage.READ);
> put(Verb.REPLICATION_FINISHED, Stage.MISC);
> *put(Verb.COUNTER_MUTATION, Stage.MUTATION);**
> put(Verb.SNAPSHOT, Stage.MISC);
> put(Verb.ECHO, Stage.GOSSIP);
> put(Verb.UNUSED_1, Stage.INTERNAL_RESPONSE);
> put(Verb.UNUSED_2, Stage.INTERNAL_RESPONSE);
> put(Verb.UNUSED_3, Stage.INTERNAL_RESPONSE);
> }};



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10605) MessagingService: COUNTER_MUTATION verb is associated with MUTATION stage

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10605:
-
Reproduced In: 3.x  (was: 2.1.11)

> MessagingService: COUNTER_MUTATION verb is associated with MUTATION stage
> -
>
> Key: CASSANDRA-10605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10605
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Anubhav Kale
> Fix For: 2.1.10
>
> Attachments: trunk-10605.patch
>
>
> In MessagingService.java, I see that the verb COUNTER_MUTATION is associated 
> with COUNTER_MUTATION stage first, and later with MUTATION stage. We should 
> remove appropriate entry.
> I marked this as Major because I think the stage assignment is wrong.
> What should this be assigned to ?
> public static final EnumMap verbStages = new 
> EnumMap(MessagingService.Verb.class)
> {{
> put(Verb.MUTATION, Stage.MUTATION);
> put(Verb.COUNTER_MUTATION, Stage.COUNTER_MUTATION);*
> put(Verb.READ_REPAIR, Stage.MUTATION);
> put(Verb.HINT, Stage.MUTATION);
> put(Verb.TRUNCATE, Stage.MUTATION);
> put(Verb.PAXOS_PREPARE, Stage.MUTATION);
> put(Verb.PAXOS_PROPOSE, Stage.MUTATION);
> put(Verb.PAXOS_COMMIT, Stage.MUTATION);
> put(Verb.BATCH_STORE, Stage.MUTATION);
> put(Verb.BATCH_REMOVE, Stage.MUTATION);
> put(Verb.READ, Stage.READ);
> put(Verb.RANGE_SLICE, Stage.READ);
> put(Verb.INDEX_SCAN, Stage.READ);
> put(Verb.PAGED_RANGE, Stage.READ);
> put(Verb.REQUEST_RESPONSE, Stage.REQUEST_RESPONSE);
> put(Verb.INTERNAL_RESPONSE, Stage.INTERNAL_RESPONSE);
> put(Verb.STREAM_REPLY, Stage.MISC); // actually handled by 
> FileStreamTask and streamExecutors
> put(Verb.STREAM_REQUEST, Stage.MISC);
> put(Verb.REPLICATION_FINISHED, Stage.MISC);
> put(Verb.SNAPSHOT, Stage.MISC);
> put(Verb.TREE_REQUEST, Stage.ANTI_ENTROPY);
> put(Verb.TREE_RESPONSE, Stage.ANTI_ENTROPY);
> put(Verb.STREAMING_REPAIR_REQUEST, Stage.ANTI_ENTROPY);
> put(Verb.STREAMING_REPAIR_RESPONSE, Stage.ANTI_ENTROPY);
> put(Verb.REPAIR_MESSAGE, Stage.ANTI_ENTROPY);
> put(Verb.GOSSIP_DIGEST_ACK, Stage.GOSSIP);
> put(Verb.GOSSIP_DIGEST_ACK2, Stage.GOSSIP);
> put(Verb.GOSSIP_DIGEST_SYN, Stage.GOSSIP);
> put(Verb.GOSSIP_SHUTDOWN, Stage.GOSSIP);
> put(Verb.DEFINITIONS_UPDATE, Stage.MIGRATION);
> put(Verb.SCHEMA_CHECK, Stage.MIGRATION);
> put(Verb.MIGRATION_REQUEST, Stage.MIGRATION);
> put(Verb.INDEX_SCAN, Stage.READ);
> put(Verb.REPLICATION_FINISHED, Stage.MISC);
> *put(Verb.COUNTER_MUTATION, Stage.MUTATION);**
> put(Verb.SNAPSHOT, Stage.MISC);
> put(Verb.ECHO, Stage.GOSSIP);
> put(Verb.UNUSED_1, Stage.INTERNAL_RESPONSE);
> put(Verb.UNUSED_2, Stage.INTERNAL_RESPONSE);
> put(Verb.UNUSED_3, Stage.INTERNAL_RESPONSE);
> }};



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8895) Compressed sstables should only compress if the win is above a certain threshold, and should use a variable block size

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-8895:
---
Assignee: (was: Paulo Motta)

> Compressed sstables should only compress if the win is above a certain 
> threshold, and should use a variable block size
> --
>
> Key: CASSANDRA-8895
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8895
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>  Labels: performance
> Fix For: 3.x
>
>
> On performing a flush to disk, we should assess if the data we're flushing 
> will actually be substantively compressed, and how large the page should be 
> to get optimal compression ratio versus read latency. Decompressing 64Kb 
> chunks is wasteful when reading small records.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10614) AsertionError while flushing memtables

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10614:

Component/s: Local Write-Read Paths

> AsertionError while flushing memtables
> --
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10614) AsertionError while flushing memtables

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978735#comment-14978735
 ] 

Sylvain Lebresne commented on CASSANDRA-10614:
--

>From the assertion, it sounds like an empty row had made its way in the 
>memtable. Maybe the MV code can generate an empty row in some conditions?

> AsertionError while flushing memtables
> --
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: T Jake Luciani
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10092) Generalize PerRowSecondaryIndex validation

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978759#comment-14978759
 ] 

Sam Tunnicliffe commented on CASSANDRA-10092:
-

[~adelapena] thanks for the updated patch. I appreciate the thrift API is a bit 
esoteric, so thanks for persevering with it. I've backported the v3 patch to 
2.1 and pushed both 2.1 & 2.2 branches to my local repo for CI. Once those test 
results are clean I'll commit. Thanks again.

||2.1||2.2||
|[branch|https://github.com/beobal/cassandra/tree/10092-2.1]|[branch|https://github.com/beobal/cassandra/tree/10092-2.2]|
|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.2-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.1-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.2-dtest/]|

> Generalize PerRowSecondaryIndex validation
> --
>
> Key: CASSANDRA-10092
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10092
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
>  Labels: 2i, secondary_index, validation
> Fix For: 2.1.x, 2.2.x
>
> Attachments: CASSANDRA-10092_v2.patch, CASSANDRA-10092_v3.patch, 
> improve_2i_validation.patch
>
>
> Index validation is currently done in a per-cell basis. However, per-row 
> secondary index developers can be interested in validating all the written 
> columns at once, because some implementations need to check the validity of a 
> row write by comparing some column values against others. For example, a per 
> row 2i implementation indexing time ranges (composed by a start date column 
> and an end date column) should check that the start date is before the stop 
> date.
> I'm attaching a patch adding a new method to {{PerRowSecondaryIndex}}:
> {code:java}
> public void validate(ByteBuffer key, ColumnFamily cf) throws 
> InvalidRequestException {}
> {code}
> and a new method to {{SecondaryIndexManager}}:
> {code:java}
> public void validateRowLevelIndexes(ByteBuffer key, ColumnFamily cf) throws 
> InvalidRequestException
>   {
>   for (SecondaryIndex index : rowLevelIndexMap.values())
>   {
>   ((PerRowSecondaryIndex) index).validate(key, cf);
>   }
>   }
> {code}
> This method is invoked in CQL {{UpdateStatement#validateIndexedColumns}}. 
> This way, {{PerRowSecondaryIndex}} could perform complex write validation.
> I have tried to do the patch in the least invasive way possible. Maybe the 
> current method {{SecondaryIndex#validate(ByteBuffer, Cell)}} should be moved 
> to {{PerColumnSecondaryIndex}}, and the {{InvalidRequestException}} that 
> informs about the particular 64k limitation should be thrown by 
> {{AbstractSimplePerColumnSecondaryIndex}}. However, given the incoming  
> [CASSANDRA-9459|https://issues.apache.org/jira/browse/CASSANDRA-9459], I 
> think that the proposed patch is more than enough to provide rich validation 
> features to 2i implementations based on 2.1.x and 2.2.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8927) Mark libjna-java + libjna-jni as incompatible in debian package

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978967#comment-14978967
 ] 

Robert Stupp commented on CASSANDRA-8927:
-

ping [~mshuler]

> Mark libjna-java + libjna-jni as incompatible in debian package
> ---
>
> Key: CASSANDRA-8927
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8927
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
> Environment: Debian
>Reporter: Robert Stupp
>Assignee: Michael Shuler
> Fix For: 2.1.x
>
>
> Current Debian (Wheezy) might bring {{libjna-java}} in version 3.2.7-4, which 
> has incompatible {{libjnadispatch.so}} because since C* 2.1 we use JNA 4.0.0 
> (the native stuff changed):
> jna.jar includes all binaries for all supported platforms - so there's no 
> need for libjna installed separately.
> Since CASSANDRA-8714 has been committed, the incompatibility manifests in 
> {{java.lang.NoClassDefFoundError: Could not initialize class 
> com.sun.jna.Native}} (which is caused by outdated libjna-java installed via 
> apt).
> Note: Debian jessie adds new package {{libjna-jni}} (4.1.0-1) in addition to 
> {{libjna-java}} (4.1.0-1) - both contain the {{libjnidispatch.so}}. Although 
> these seem to work, we might hit the same issue when there's a need to 
> upgrade JNA to 4.2.x sometime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[samt]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a88d3e89
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a88d3e89
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a88d3e89

Branch: refs/heads/trunk
Commit: a88d3e89628df6fb9e6ff3332c546d92168284f1
Parents: 74ee668 8210075
Author: Sam Tunnicliffe 
Authored: Wed Oct 28 17:26:43 2015 +
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 17:26:43 2015 +

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/index/Index.java  | 11 ++
 .../cassandra/index/SecondaryIndexManager.java  | 16 +++-
 .../index/internal/CassandraIndex.java  |  1 -
 .../apache/cassandra/index/CustomIndexTest.java | 40 
 .../index/internal/CassandraIndexTest.java  |  9 +
 6 files changed, 66 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a88d3e89/CHANGES.txt
--
diff --cc CHANGES.txt
index 1b6ede7,c945bd2..114e2aa
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
   * Fix thrift cas operations with defined columns (CASSANDRA-10576)
   * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
   * Fix thrift get() queries with defined columns (CASSANDRA-10586)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a88d3e89/src/java/org/apache/cassandra/index/Index.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a88d3e89/test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
--

[jira] [Commented] (CASSANDRA-4782) Commitlog not replayed after restart

[Robert Coli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978905#comment-14978905
 ] 

Robert Coli commented on CASSANDRA-4782:


[~htoulan] :

1) JIRA is not a support forum, generally. You would get a better response from 
the #cassandra IRC channel on freenode or the cassandra-user@ mailing list.

2) Cassandra 1.1.0 is an extremely old and extremely broken version. You should 
update to at least the final released version of the 1.1 line. Much better 
would be (first 1.2.x and then) at least the most recent 2.0.x version.

3) No commit log replay bug, including this one, should be capable of losing 
data if you write with a sufficient ConsistencyLevel and Replication Factor and 
repair regularly.

> Commitlog not replayed after restart
> 
>
> Key: CASSANDRA-4782
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4782
> Project: Cassandra
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fabien Rousseau
>Assignee: Jonathan Ellis
>Priority: Critical
> Fix For: 1.1.6
>
> Attachments: 4782.txt
>
>
> It seems that there are two corner cases where commitlog is not replayed 
> after a restart :
>  - After a reboot of a server + restart of cassandra (1.1.0 to 1.1.4)
>  - After doing an upgrade from cassandra 1.1.X to cassandra 1.1.5
> This is due to the fact that the commitlog segment id should always be an  
> incrementing number (see this condition : 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L247
>  )
> But this assertion can be broken :
> In the first case, it is generated by System.nanoTime() but it seems that 
> System.nanoTime() is using the boot time as the base/reference (at least on 
> java6 & linux), thus after a reboot, System.nanoTime() can return a lower 
> number than before the reboot (and the javadoc says the reference is a 
> relative point in time...)
> In the second case, this was introduced by #4601 (which changes 
> System.nanoTime() by System.currentTimeMillis() thus people starting with 
> 1.1.5 are safe)
> This could explain the following tickets : #4741 and #4481



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Fix Thrift CAS operations for columns with defined metadata

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 3615da583 -> 5034f4495
  refs/heads/trunk a4fe865d4 -> 74ee6684c


Fix Thrift CAS operations for columns with defined metadata

Patch by Sam Tunnicliffe; reviewed by Sylvain Lebresne for
CASSANDRA-10576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5034f449
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5034f449
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5034f449

Branch: refs/heads/cassandra-3.0
Commit: 5034f449575a4648f67b50c1ff535518ce2f00e9
Parents: 3615da5
Author: Sam Tunnicliffe 
Authored: Fri Oct 23 13:51:29 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 17:11:04 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/thrift/CassandraServer.java   | 40 +---
 2 files changed, 28 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5034f449/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c51cb51..4ead854 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix thrift cas operations with defined columns (CASSANDRA-10576)
  * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
  * Fix thrift get() queries with defined columns (CASSANDRA-10586)
  * Fix marking of indexes as built and removed (CASSANDRA-10601)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5034f449/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index 2a3f73d..02c0871 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -945,15 +945,11 @@ public class CassandraServer implements Cassandra.Iface
 // Indexed column values cannot be larger than 64K.  See 
CASSANDRA-3057/4240 for more details
 
Keyspace.open(metadata.ksName).getColumnFamilyStore(metadata.cfName).indexManager.validate(partitionUpdates);
 
-FilteredPartition partitionExpected = null;
-if (!expected.isEmpty())
-partitionExpected = 
FilteredPartition.create(LegacyLayout.toRowIterator(metadata, dk, 
toLegacyCells(metadata, expected, nowInSec).iterator(), nowInSec));
-
 schedule(DatabaseDescriptor.getWriteRpcTimeout());
 try (RowIterator result = StorageProxy.cas(cState.getKeyspace(),
column_family,
dk,
-   new 
ThriftCASRequest(partitionExpected, partitionUpdates),
+   new 
ThriftCASRequest(toLegacyCells(metadata, expected, nowInSec), partitionUpdates, 
nowInSec),

ThriftConversion.fromThrift(serial_consistency_level),

ThriftConversion.fromThrift(commit_consistency_level),
cState))
@@ -2509,21 +2505,22 @@ public class CassandraServer implements Cassandra.Iface
 {
 private final CFMetaData metadata;
 private final DecoratedKey key;
-
-private final FilteredPartition expected;
+private final List expected;
 private final PartitionUpdate updates;
+private final int nowInSec;
 
-private ThriftCASRequest(FilteredPartition expected, PartitionUpdate 
updates)
+private ThriftCASRequest(List expected, 
PartitionUpdate updates, int nowInSec)
 {
 this.metadata = updates.metadata();
 this.key = updates.partitionKey();
 this.expected = expected;
 this.updates = updates;
+this.nowInSec = nowInSec;
 }
 
 public SinglePartitionReadCommand readCommand(int nowInSec)
 {
-if (expected == null || expected.isEmpty())
+if (expected.isEmpty())
 {
 // We want to know if the partition exists, so just fetch a 
single cell.
 ClusteringIndexSliceFilter filter = new 
ClusteringIndexSliceFilter(Slices.ALL, false);
@@ -2533,9 +2530,13 @@ public class CassandraServer implements Cassandra.Iface
 
 // Gather the clustering for the expected values and query those.
 BTreeSet.Builder clusterings = 

[2/3] cassandra git commit: Fix Thrift CAS operations for columns with defined metadata

[samt]

Fix Thrift CAS operations for columns with defined metadata

Patch by Sam Tunnicliffe; reviewed by Sylvain Lebresne for
CASSANDRA-10576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5034f449
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5034f449
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5034f449

Branch: refs/heads/trunk
Commit: 5034f449575a4648f67b50c1ff535518ce2f00e9
Parents: 3615da5
Author: Sam Tunnicliffe 
Authored: Fri Oct 23 13:51:29 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 17:11:04 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/thrift/CassandraServer.java   | 40 +---
 2 files changed, 28 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5034f449/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c51cb51..4ead854 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix thrift cas operations with defined columns (CASSANDRA-10576)
  * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
  * Fix thrift get() queries with defined columns (CASSANDRA-10586)
  * Fix marking of indexes as built and removed (CASSANDRA-10601)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5034f449/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java 
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index 2a3f73d..02c0871 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -945,15 +945,11 @@ public class CassandraServer implements Cassandra.Iface
 // Indexed column values cannot be larger than 64K.  See 
CASSANDRA-3057/4240 for more details
 
Keyspace.open(metadata.ksName).getColumnFamilyStore(metadata.cfName).indexManager.validate(partitionUpdates);
 
-FilteredPartition partitionExpected = null;
-if (!expected.isEmpty())
-partitionExpected = 
FilteredPartition.create(LegacyLayout.toRowIterator(metadata, dk, 
toLegacyCells(metadata, expected, nowInSec).iterator(), nowInSec));
-
 schedule(DatabaseDescriptor.getWriteRpcTimeout());
 try (RowIterator result = StorageProxy.cas(cState.getKeyspace(),
column_family,
dk,
-   new 
ThriftCASRequest(partitionExpected, partitionUpdates),
+   new 
ThriftCASRequest(toLegacyCells(metadata, expected, nowInSec), partitionUpdates, 
nowInSec),

ThriftConversion.fromThrift(serial_consistency_level),

ThriftConversion.fromThrift(commit_consistency_level),
cState))
@@ -2509,21 +2505,22 @@ public class CassandraServer implements Cassandra.Iface
 {
 private final CFMetaData metadata;
 private final DecoratedKey key;
-
-private final FilteredPartition expected;
+private final List expected;
 private final PartitionUpdate updates;
+private final int nowInSec;
 
-private ThriftCASRequest(FilteredPartition expected, PartitionUpdate 
updates)
+private ThriftCASRequest(List expected, 
PartitionUpdate updates, int nowInSec)
 {
 this.metadata = updates.metadata();
 this.key = updates.partitionKey();
 this.expected = expected;
 this.updates = updates;
+this.nowInSec = nowInSec;
 }
 
 public SinglePartitionReadCommand readCommand(int nowInSec)
 {
-if (expected == null || expected.isEmpty())
+if (expected.isEmpty())
 {
 // We want to know if the partition exists, so just fetch a 
single cell.
 ClusteringIndexSliceFilter filter = new 
ClusteringIndexSliceFilter(Slices.ALL, false);
@@ -2533,9 +2530,13 @@ public class CassandraServer implements Cassandra.Iface
 
 // Gather the clustering for the expected values and query those.
 BTreeSet.Builder clusterings = 
BTreeSet.builder(metadata.comparator);
-for (Row row : expected)
+FilteredPartition expectedPartition =
+

[jira] [Assigned] (CASSANDRA-10614) AssertionError while flushing memtables

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-10614:
---

Assignee: Tyler Hobbs  (was: T Jake Luciani)

> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[samt]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/74ee6684
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/74ee6684
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/74ee6684

Branch: refs/heads/trunk
Commit: 74ee6684ca1f93ccb737b68bc401dba23045610f
Parents: a4fe865 5034f44
Author: Sam Tunnicliffe 
Authored: Wed Oct 28 17:15:10 2015 +
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 17:15:10 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/thrift/CassandraServer.java   | 40 +---
 2 files changed, 28 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/74ee6684/CHANGES.txt
--
diff --cc CHANGES.txt
index 29117e0,4ead854..1b6ede7
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * Fix thrift cas operations with defined columns (CASSANDRA-10576)
   * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
   * Fix thrift get() queries with defined columns (CASSANDRA-10586)
   * Fix marking of indexes as built and removed (CASSANDRA-10601)

[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978789#comment-14978789
 ] 

Tyler Hobbs commented on CASSANDRA-10614:
-

It looks like this was introduced by CASSANDRA-9664, and is likely the cause of 
CASSANDRA-10609.

> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: T Jake Luciani
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10070) Automatic repair scheduling

[Avinash Mandava (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978802#comment-14978802
 ] 

Avinash Mandava commented on CASSANDRA-10070:
-

Right now streaming between Cassandra versions isn't recommended, but I'm 
wondering how we would upgrade to a new version with automatic repairs running. 
If I'm doing a rolling upgrade, right now I have to stop the repair process to 
prevent streaming between nodes and then upgrade, and then resume the repair 
process. But if we are thinking of including automatic repairs, might it be 
valuable to allow people to keep the repair process going while they upgrade? I 
can see how an upgrade is infrequent enough for this suggestion to be overkill, 
but curious what people think.

> Automatic repair scheduling
> ---
>
> Key: CASSANDRA-10070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10070
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Minor
> Fix For: 3.x
>
>
> Scheduling and running repairs in a Cassandra cluster is most often a 
> required task, but this can both be hard for new users and it also requires a 
> bit of manual configuration. There are good tools out there that can be used 
> to simplify things, but wouldn't this be a good feature to have inside of 
> Cassandra? To automatically schedule and run repairs, so that when you start 
> up your cluster it basically maintains itself in terms of normal 
> anti-entropy, with the possibility for manual configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: On CFS::reload, execute the reload task for each secondary index

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 5034f4495 -> 8210075c5
  refs/heads/trunk 74ee6684c -> a88d3e896


On CFS::reload, execute the reload task for each secondary index

Patch by Sam Tunnicliffe; reviewed by Tyler Hobbs for CASSANDRA-10604


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8210075c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8210075c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8210075c

Branch: refs/heads/cassandra-3.0
Commit: 8210075c57af696e78000e8cd0019737b2373875
Parents: 5034f44
Author: Sam Tunnicliffe 
Authored: Wed Oct 28 13:16:15 2015 +
Committer: Sam Tunnicliffe 
Committed: Wed Oct 28 17:23:57 2015 +

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/index/Index.java  | 11 ++
 .../cassandra/index/SecondaryIndexManager.java  | 16 +++-
 .../index/internal/CassandraIndex.java  |  1 -
 .../apache/cassandra/index/CustomIndexTest.java | 40 
 .../index/internal/CassandraIndexTest.java  |  9 +
 6 files changed, 66 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8210075c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4ead854..c945bd2 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
  * Fix thrift cas operations with defined columns (CASSANDRA-10576)
  * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
  * Fix thrift get() queries with defined columns (CASSANDRA-10586)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8210075c/src/java/org/apache/cassandra/index/Index.java
--
diff --git a/src/java/org/apache/cassandra/index/Index.java 
b/src/java/org/apache/cassandra/index/Index.java
index 0f4ecbd..b6c12a9 100644
--- a/src/java/org/apache/cassandra/index/Index.java
+++ b/src/java/org/apache/cassandra/index/Index.java
@@ -296,9 +296,20 @@ public interface Index
  * Listener for processing events emitted during a single partition update.
  * Instances of this are responsible for applying modifications to the 
index in response to a single update
  * operation on a particular partition of the base table.
+ *
  * That update may be generated by the normal write path, by iterating 
SSTables during streaming operations or when
  * building or rebuilding an index from source. Updates also occur during 
compaction when multiple versions of a
  * source partition from different SSTables are merged.
+ *
+ * Implementations should not make assumptions about resolution or 
filtering of the partition update being
+ * processed. That is to say that it is possible for an Indexer instance 
to receive notification of a
+ * PartitionDelete or RangeTombstones which shadow a Row it then receives 
via insertRow/updateRow.
+ *
+ * It is important to note that the only ordering guarantee made for the 
methods here is that the first call will
+ * be to begin() and the last call to finish(). The other methods may be 
called to process update events in any
+ * order. This can also include duplicate calls, in cases where a memtable 
partition is under contention from
+ * several updates. In that scenario, the same set of events may be 
delivered to the Indexer as memtable update
+ * which failed due to contention is re-applied.
  */
 public interface Indexer
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8210075c/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --git a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java 
b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 3ed9714..92b04fe 100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@ -142,17 +142,11 @@ public class SecondaryIndexManager implements 
IndexRegistry
 
 private Future reloadIndex(IndexMetadata indexDef)
 {
-// if the index metadata has changed, reload the index
-IndexMetadata registered = 
indexes.get(indexDef.name).getIndexMetadata();
-if (!registered.equals(indexDef))
-{
-Index index = indexes.remove(registered.name);
-index.register(this);
-return 
blockingExecutor.submit(index.getMetadataReloadTask(indexDef));
-}
-
-