[jira] [Commented] (CASSANDRA-12172) Fail to bootstrap new node.

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372209#comment-15372209
 ] 

Joel Knighton commented on CASSANDRA-12172:
---

I'm not sure this is a bug - if so, we'll need more information to resolve it.

This error message is correct. Cassandra will try to bootstrap from the replica 
being replaced unless you advise it otherwise "-Dc
assandra.consistent.rangemovement=false". 

Cassandra's failure detection builds up a statistical estimate based on gossip 
updates as to whether a node is up or down. It may be that you have an 
unreliable network and need to tune the phi conviction threshold appropriately 
- 
https://github.com/apache/cassandra/blob/91392edbe812c722adcf35cf167bf400d25dc99a/conf/cassandra.yaml#L855.
 Otherwise, it may be the case that some behavior on the hosts being marked 
down is preventing them from gossiping/performing other tasks, such as a long 
GC pause. In this sense, the bug is not in the failure detection but in some 
other component.

We could get a better perspective on this with trace/debug level logs from the 
bootstrapping node and also a node marked down at the time of bootstrap.

> Fail to bootstrap new node.
> ---
>
> Key: CASSANDRA-12172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12172
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dikang Gu
>
> When I try to bootstrap new node in the cluster, sometimes it failed because 
> of following exceptions.
> {code}
> 2016-07-12_05:14:55.58509 INFO  05:14:55 [main]: JOINING: Starting to 
> bootstrap...
> 2016-07-12_05:14:56.07491 INFO  05:14:56 [GossipTasks:1]: InetAddress 
> /2401:db00:2011:50c7:face:0:9:0 is now DOWN
> 2016-07-12_05:14:56.32219 Exception (java.lang.RuntimeException) encountered 
> during startup: A node required to move the data consistently is down 
> (/2401:db00:2011:50c7:face:0:9:0). If you wish to move the data from a 
> potentially inconsis
> tent replica, restart the node with -Dcassandra.consistent.rangemovement=false
> 2016-07-12_05:14:56.32582 ERROR 05:14:56 [main]: Exception encountered during 
> startup
> 2016-07-12_05:14:56.32583 java.lang.RuntimeException: A node required to move 
> the data consistently is down (/2401:db00:2011:50c7:face:0:9:0). If you wish 
> to move the data from a potentially inconsistent replica, restart the node 
> with -Dc
> assandra.consistent.rangemovement=false
> 2016-07-12_05:14:56.32584   at 
> org.apache.cassandra.dht.RangeStreamer.getAllRangesWithStrictSourcesFor(RangeStreamer.java:264)
>  ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584   at 
> org.apache.cassandra.dht.RangeStreamer.addRanges(RangeStreamer.java:147) 
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584   at 
> org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:82) 
> ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1230)
>  ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32584   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:924)
>  ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32585   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:709)
>  ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32585   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:585)
>  ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32585   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
> [apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32586   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516)
>  [apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32586   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625) 
> [apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
> 2016-07-12_05:14:56.32730 WARN  05:14:56 [StorageServiceShutdownHook]: No 
> local state or state is in silent shutdown, not announcing shutdown
> {code}
> Here are more logs: 
> https://gist.github.com/DikangGu/c6a83eafdbc091250eade4a3bddcc40b
> I'm pretty sure there are no DOWN nodes or restarted nodes in the cluster, 
> but I still see a lot of nodes UP and DOWN in the gossip log, which failed 
> the bootstrap at the end, is this a known 

[jira] [Updated] (CASSANDRA-10805) Additional Compaction Logging

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-10805:
-
Labels: doc-impacting  (was: )

> Additional Compaction Logging
> -
>
> Key: CASSANDRA-10805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10805
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction, Observability
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
>Priority: Minor
>  Labels: doc-impacting
> Fix For: 3.6
>
>
> Currently, viewing the results of past compactions requires parsing the log 
> and looking at the compaction history system table, which doesn't have 
> information about, for example, flushed sstables not previously compacted.
> This is a proposal to extend the information captured for compaction. 
> Initially, this would be done through a JMX call, but if it proves to be 
> useful and not much overhead, it might be a feature that could be enabled for 
> the compaction strategy all the time.
> Initial log information would include:
> - The compaction strategy type controlling each column family
> - The set of sstables included in each compaction strategy
> - Information about flushes and compactions, including times and all involved 
> sstables
> - Information about sstables, including generation, size, and tokens
> - Any additional metadata the strategy wishes to add to a compaction or an 
> sstable, like the level of an sstable or the type of compaction being 
> performed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12172) Fail to bootstrap new node.

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-12172:
--
Description: 
When I try to bootstrap new node in the cluster, sometimes it failed because of 
following exceptions.

{code}
2016-07-12_05:14:55.58509 INFO  05:14:55 [main]: JOINING: Starting to 
bootstrap...
2016-07-12_05:14:56.07491 INFO  05:14:56 [GossipTasks:1]: InetAddress 
/2401:db00:2011:50c7:face:0:9:0 is now DOWN
2016-07-12_05:14:56.32219 Exception (java.lang.RuntimeException) encountered 
during startup: A node required to move the data consistently is down 
(/2401:db00:2011:50c7:face:0:9:0). If you wish to move the data from a 
potentially inconsis
tent replica, restart the node with -Dcassandra.consistent.rangemovement=false
2016-07-12_05:14:56.32582 ERROR 05:14:56 [main]: Exception encountered during 
startup
2016-07-12_05:14:56.32583 java.lang.RuntimeException: A node required to move 
the data consistently is down (/2401:db00:2011:50c7:face:0:9:0). If you wish to 
move the data from a potentially inconsistent replica, restart the node with -Dc
assandra.consistent.rangemovement=false
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.RangeStreamer.getAllRangesWithStrictSourcesFor(RangeStreamer.java:264)
 ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.RangeStreamer.addRanges(RangeStreamer.java:147) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:82) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1230) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:924)
 ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32585   at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:709) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32585   at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:585) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32585   at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32586   at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516) 
[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32586   at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625) 
[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32730 WARN  05:14:56 [StorageServiceShutdownHook]: No local 
state or state is in silent shutdown, not announcing shutdown
{code}

Here are more logs: 
https://gist.github.com/DikangGu/c6a83eafdbc091250eade4a3bddcc40b

I'm pretty sure there are no DOWN nodes or restarted nodes in the cluster, but 
I still see a lot of nodes UP and DOWN in the gossip log, which failed the 
bootstrap at the end, is this a known bug?

  was:
When I try to bootstrap new node in the cluster, sometimes it failed because of 
following exceptions.


2016-07-12_05:14:55.58509 INFO  05:14:55 [main]: JOINING: Starting to 
bootstrap...
2016-07-12_05:14:56.07491 INFO  05:14:56 [GossipTasks:1]: InetAddress 
/2401:db00:2011:50c7:face:0:9:0 is now DOWN
2016-07-12_05:14:56.32219 Exception (java.lang.RuntimeException) encountered 
during startup: A node required to move the data consistently is down 
(/2401:db00:2011:50c7:face:0:9:0). If you wish to move the data from a 
potentially inconsis
tent replica, restart the node with -Dcassandra.consistent.rangemovement=false
2016-07-12_05:14:56.32582 ERROR 05:14:56 [main]: Exception encountered during 
startup
2016-07-12_05:14:56.32583 java.lang.RuntimeException: A node required to move 
the data consistently is down (/2401:db00:2011:50c7:face:0:9:0). If you wish to 
move the data from a potentially inconsistent replica, restart the node with -Dc
assandra.consistent.rangemovement=false
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.RangeStreamer.getAllRangesWithStrictSourcesFor(RangeStreamer.java:264)
 ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.RangeStreamer.addRanges(RangeStreamer.java:147) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]

[jira] [Created] (CASSANDRA-12172) Fail to bootstrap new node.

[Dikang Gu (JIRA)]

Dikang Gu created CASSANDRA-12172:
-

 Summary: Fail to bootstrap new node.
 Key: CASSANDRA-12172
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12172
 Project: Cassandra
  Issue Type: Bug
Reporter: Dikang Gu


When I try to bootstrap new node in the cluster, sometimes it failed because of 
following exceptions.


2016-07-12_05:14:55.58509 INFO  05:14:55 [main]: JOINING: Starting to 
bootstrap...
2016-07-12_05:14:56.07491 INFO  05:14:56 [GossipTasks:1]: InetAddress 
/2401:db00:2011:50c7:face:0:9:0 is now DOWN
2016-07-12_05:14:56.32219 Exception (java.lang.RuntimeException) encountered 
during startup: A node required to move the data consistently is down 
(/2401:db00:2011:50c7:face:0:9:0). If you wish to move the data from a 
potentially inconsis
tent replica, restart the node with -Dcassandra.consistent.rangemovement=false
2016-07-12_05:14:56.32582 ERROR 05:14:56 [main]: Exception encountered during 
startup
2016-07-12_05:14:56.32583 java.lang.RuntimeException: A node required to move 
the data consistently is down (/2401:db00:2011:50c7:face:0:9:0). If you wish to 
move the data from a potentially inconsistent replica, restart the node with -Dc
assandra.consistent.rangemovement=false
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.RangeStreamer.getAllRangesWithStrictSourcesFor(RangeStreamer.java:264)
 ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.RangeStreamer.addRanges(RangeStreamer.java:147) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:82) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1230) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32584   at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:924)
 ~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32585   at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:709) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32585   at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:585) 
~[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32585   at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32586   at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516) 
[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32586   at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625) 
[apache-cassandra-2.2.5+git20160315.c29948b.jar:2.2.5+git20160315.c29948b]
2016-07-12_05:14:56.32730 WARN  05:14:56 [StorageServiceShutdownHook]: No local 
state or state is in silent shutdown, not announcing shutdown


Here are more logs: 
https://gist.github.com/DikangGu/c6a83eafdbc091250eade4a3bddcc40b

I'm pretty sure there are no DOWN nodes or restarted nodes in the cluster, but 
I still see a lot of nodes UP and DOWN in the gossip log, which failed the 
bootstrap at the end, is this a known bug?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: fix SSTableSizeSummer's file skipping logic

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk de73b5c7b -> 91392edbe


fix SSTableSizeSummer's file skipping logic


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/91392edb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/91392edb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/91392edb

Branch: refs/heads/trunk
Commit: 91392edbe812c722adcf35cf167bf400d25dc99a
Parents: de73b5c
Author: Dave Brosius 
Authored: Tue Jul 12 00:20:40 2016 -0400
Committer: Dave Brosius 
Committed: Tue Jul 12 00:20:40 2016 -0400

--
 src/java/org/apache/cassandra/db/Directories.java | 10 --
 1 file changed, 4 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/91392edb/src/java/org/apache/cassandra/db/Directories.java
--
diff --git a/src/java/org/apache/cassandra/db/Directories.java 
b/src/java/org/apache/cassandra/db/Directories.java
index 87527e8..a83c845 100644
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@ -17,8 +17,6 @@
  */
 package org.apache.cassandra.db;
 
-import static com.google.common.collect.Sets.newHashSet;
-
 import java.io.File;
 import java.io.FileFilter;
 import java.io.IOError;
@@ -1014,14 +1012,14 @@ public class Directories
 }
 
 @Override
-public boolean isAcceptable(Path file)
+public boolean isAcceptable(Path path)
 {
-String fileName = file.toFile().getName();
-Pair pair = 
SSTable.tryComponentFromFilename(file.getParent().toFile(), fileName);
+File file = path.toFile();
+Pair pair = 
SSTable.tryComponentFromFilename(path.getParent().toFile(), file.getName());
 return pair != null
 && pair.left.ksname.equals(metadata.ksName)
 && pair.left.cfname.equals(metadata.cfName)
-&& !toSkip.contains(fileName);
+&& !toSkip.contains(file);
 }
 }
 }

cassandra git commit: use precomputed end ClusteringBound

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 019d43734 -> de73b5c7b


use precomputed end ClusteringBound


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/de73b5c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/de73b5c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/de73b5c7

Branch: refs/heads/trunk
Commit: de73b5c7bef402782ff775fba772cb3f870bbc16
Parents: 019d437
Author: Dave Brosius 
Authored: Tue Jul 12 00:08:53 2016 -0400
Committer: Dave Brosius 
Committed: Tue Jul 12 00:08:53 2016 -0400

--
 src/java/org/apache/cassandra/db/RangeTombstoneList.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/de73b5c7/src/java/org/apache/cassandra/db/RangeTombstoneList.java
--
diff --git a/src/java/org/apache/cassandra/db/RangeTombstoneList.java 
b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
index c60b774..716213d 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstoneList.java
+++ b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
@@ -549,7 +549,7 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 ClusteringBound newEnd = start.invert();
 if (!Slice.isEmpty(comparator, starts[i], newEnd))
 {
-addInternal(i, starts[i], start.invert(), 
markedAts[i], delTimes[i]);
+addInternal(i, starts[i], newEnd, markedAts[i], 
delTimes[i]);
 i++;
 setInternal(i, start, ends[i], markedAts[i], 
delTimes[i]);
 }

cassandra git commit: push down indexFileLength calc, to where it's used

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9fed08449 -> 019d43734


push down indexFileLength calc, to where it's used


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/019d4373
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/019d4373
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/019d4373

Branch: refs/heads/trunk
Commit: 019d43734d30499242af621541cf1c48958046e3
Parents: 9fed084
Author: Dave Brosius 
Authored: Tue Jul 12 00:00:04 2016 -0400
Committer: Dave Brosius 
Committed: Tue Jul 12 00:00:04 2016 -0400

--
 src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/019d4373/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
index 78a6825..fc0849f 100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
@@ -752,11 +752,11 @@ public abstract class SSTableReader extends SSTable 
implements SelfRefCounted

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372114#comment-15372114
 ] 

Stefania commented on CASSANDRA-9318:
-

bq. But can that really happen? ResponseVerbHandler returns before incrementing 
back-pressure if the callback is null (i.e. expired), and OutboundTcpConnection 
doesn't even send outbound messages if they're timed out, or am I missing 
something?

You're correct, we won't count twice because the callback is already null. 
However, this raises another point, if a message expires before it is sent, we 
consider this negatively for that replica, since we increment the outgoing rate 
but not the incoming rate when the callback expires, and still it may have 
nothing to do with the replica if the message was not sent, it may be due to 
the coordinator dealing with too many messages.
 
bq. Again, I believe this would make enabling/disabling back-pressure via JMX 
less user friendly.

Fine, let's keep the boolean since it makes life easier for JMX.

bq. I do not think sorting replicas is what we really need, as you have to send 
the mutation to all replicas anyway. I think what you rather need is a way to 
pre-emptively fail if the write consistency level is not met by enough 
"non-overloaded" replicas, i.e.:

You're correct in that the replicas are not sorted in the write path, only in 
the read path. I confused the two yesterday. For sure we need to only fail if 
the write consistency level is not met. I also observe that if a replica has a 
low rate, then we may block when acquiring the limiter, and this will 
indirectly throttle for all following replicas, even if they were ready to 
receive mutations sooner. Therefore, even a single overloaded or slow replica 
may slow the entire write operation. Further, AbstractWriteResponseHandler sets 
the start time in the constructor, so the time spent acquiring a rate limiter 
for slow replicas counts towards the total time before the coordinator throws a 
write timeout exception. So, unless we increase the write RPC timeout or change 
the existing behavior, we may observe write timeout exceptions and, at CL.ANY, 
hints.

Also, in SP.sendToHintedEndpoints(), we should apply backpressure only if the 
destination is alive.
 
{quote}
This leaves us with two options:

Adding a new exception to the native protocol.
Reusing a different exception, with WriteFailureException and 
UnavailableException the most likely candidates.

I'm currently leaning towards the latter option.
{quote}

Let's use UnavailableException since WriteFailureException indicates a 
non-timeout failure when processing a mutation, and so it is not appropriate 
for this case. For protocol V4 we cannot change UnavailableException, but for 
V5 we should add a new parameter to it. At the moment it contains 
{{}}, we should add the number of overloaded replicas, so 
that drivers can treat the
two cases differently. Another alternative, as suggested by [~slebresne], is to 
simply consider overloaded replicas as dead and hint them, therefore throwing 
unavailable exceptions as usual, but this is slightly less accurate then 
letting clients know that some replicas were unavailable and some simply 
overloaded.
 
bq. We only need to ensure the coordinator for that specific mutation has 
back-pressure enabled, and we could do this by "marking" the MessageOut with a 
special parameter, what do you think?

Marking messages as throttled would let the replica know if backpressure was 
enabled, that's true, but it also makes the existing mechanism even more 
complex. Also, as far as I understand it, dropping mutations that have been in 
the queue for longer that the RPC write timeout is done not only to shed load 
on the replica, but also to avoid wasting resources to perform a mutation when 
the coordinator has already returned a timeout exception to the client. I think 
this still holds true regardless of backpressure. Since we cannot remove a 
timeout check in the write response handlers, I don't see how it helps to drop 
it replica side. If the message was throttled, even with cross_node_timeout 
enabled, the replica should have time to process it before the RPC write 
timeout expires, so I don't think the extra complexity is justified.

bq. If you all agree with that, I'll move forward and make that change.

To summarize, I agree with this change, provided the drivers can separate the 
two cases (node unavailable vs. node overloaded), which they will be able to do 
with V5 of the native protocol. The alternative, would be to simply consider 
overloaded replicas as dead and hint them. Further, I still have concerns 
regarding additional write timeout exceptions and whether an overloaded or slow 
replica can slow everything down. [~slebresne], [~jbellis] anything else from 
your side? I think Jonathan's proposal of bounding total outstanding requests 
to all replicas, is 

[jira] [Updated] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-9754:
--
Tester:   (was: Michael Shuler)

> Make index info heap friendly for large CQL partitions
> --
>
> Key: CASSANDRA-9754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9754
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Michael Kjellman
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 9754_part1-v1.diff, 9754_part2-v1.diff
>
>
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects 
> are IndexInfo and its ByteBuffers. This is specially bad in endpoints with 
> large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K 
> IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for 
> GC. Can this be improved by not creating so many objects?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9608) Support Java 9

[Carlos Abad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371835#comment-15371835
 ] 

Carlos Abad commented on CASSANDRA-9608:


Carlos Abad here from the Intel Java team.

I've been able to build Cassandra with JDK9 (build 125) applying the 
modifications noted below. 
However the build only pass 70% of the Unit tests.

*At least Apache Ant version 1.9.7 is required for JDK9. Previous version are 
unable to load the JavaScript script engine manager in JDK9.

build.xml:
*Generating Bytecode for Java 9. Source is still written for Java 8. 
*Java 9 removed -Xbootclasspath/p command option. Without this option Cassandra 
will depend on the given Java classpath to include the CRC32 class
*To avoid "Annotation generator had thrown the exception. 
java.lang.NoClassDefFoundError: javax/annotation/Generated" need to add 
"-addmods java.annotations.common" to the javac task of the "build-test" target.

src/java/org/apache/cassandra/utils/Throwables.java:
*Stream.of() throws an exception now, need to be captured.

src/java/org/apache/cassandra/utils/concurrent/Locks.java:
*sun.misc.unsafe is going away. Fortunately cassandra.utils.concurrent.Locks is 
only used in one place (db/partitions/AtomicBTreePartition, see below). It'll 
be enough to modify AtomicBTreePartitions and remove this class.

src/java/org/apache/cassandra/db/partitions/AtomicBTreePartition.java:
*This is the only place where the class cassandra.utils.concurrent.Locks is 
used. We'll modify it to use Java's ReentrantLock instead.

> Support Java 9
> --
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
>  Issue Type: Task
>Reporter: Robert Stupp
>Priority: Minor
>
> This ticket is intended to group all issues found to support Java 9 in the 
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved. 
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +  
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods 
> {{monitorEnter}} + {{monitorExit}}. These methods are used by 
> {{o.a.c.utils.concurrent.Locks}} which is only used by 
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early 
> development phase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12149) NullPointerException on SELECT with SASI index

[Andrey Konstantinov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371786#comment-15371786
 ] 

Andrey Konstantinov commented on CASSANDRA-12149:
-

Thanks! Yes, this is to fetch rows sequentially by one machine. My goal was to 
fetch half of a partition by one machine and another half by the second 
machine. It seems it is impossible to do this without knowing split clustering 
key value.

> NullPointerException on SELECT with SASI index
> --
>
> Key: CASSANDRA-12149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12149
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Andrey Konstantinov
> Attachments: CASSANDRA-12149.txt
>
>
> If I execute the sequence of queries (see the attached file), Cassandra 
> aborts a connection reporting NPE on server side. SELECT query without token 
> range filter works, but does not work when token range filter is specified. 
> My intent was to issue multiple SELECT queries targeting the same single 
> partition, filtered by a column indexed by SASI, partitioning results by 
> different token ranges.
> Output from cqlsh on SELECT is the following:
> cqlsh> SELECT namespace, entity, timestamp, feature1, feature2 FROM 
> mykeyspace.myrecordtable WHERE namespace = 'ns2' AND entity = 'entity2' AND 
> feature1 > 11 AND feature1 < 31  AND token(namespace, entity) <= 
> 9223372036854775807;
> ServerError:  message="java.lang.NullPointerException">



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-9754:
--
Tester: Michael Shuler

> Make index info heap friendly for large CQL partitions
> --
>
> Key: CASSANDRA-9754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9754
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Michael Kjellman
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 9754_part1-v1.diff, 9754_part2-v1.diff
>
>
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects 
> are IndexInfo and its ByteBuffers. This is specially bad in endpoints with 
> large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K 
> IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for 
> GC. Can this be improved by not creating so many objects?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11698) dtest failure in materialized_views_test.TestMaterializedViews.clustering_column_test

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-11698:
-
Assignee: (was: Jim Witschey)

> dtest failure in 
> materialized_views_test.TestMaterializedViews.clustering_column_test
> -
>
> Key: CASSANDRA-11698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11698
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> recent failure, test has flapped before a while back.
> {noformat}
> Expecting 2 users, got 1
> {noformat}
> http://cassci.datastax.com/job/cassandra-3.0_dtest/688/testReport/materialized_views_test/TestMaterializedViews/clustering_column_test
> Failed on CassCI build cassandra-3.0_dtest #688



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11698) dtest failure in materialized_views_test.TestMaterializedViews.clustering_column_test

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey reassigned CASSANDRA-11698:


Assignee: Jim Witschey

> dtest failure in 
> materialized_views_test.TestMaterializedViews.clustering_column_test
> -
>
> Key: CASSANDRA-11698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11698
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Jim Witschey
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> recent failure, test has flapped before a while back.
> {noformat}
> Expecting 2 users, got 1
> {noformat}
> http://cassci.datastax.com/job/cassandra-3.0_dtest/688/testReport/materialized_views_test/TestMaterializedViews/clustering_column_test
> Failed on CassCI build cassandra-3.0_dtest #688



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11698) dtest failure in materialized_views_test.TestMaterializedViews.clustering_column_test

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-11698:
-
Issue Type: Bug  (was: Test)

> dtest failure in 
> materialized_views_test.TestMaterializedViews.clustering_column_test
> -
>
> Key: CASSANDRA-11698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11698
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Sean McCarthy
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> recent failure, test has flapped before a while back.
> {noformat}
> Expecting 2 users, got 1
> {noformat}
> http://cassci.datastax.com/job/cassandra-3.0_dtest/688/testReport/materialized_views_test/TestMaterializedViews/clustering_column_test
> Failed on CassCI build cassandra-3.0_dtest #688



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11698) dtest failure in materialized_views_test.TestMaterializedViews.clustering_column_test

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-11698:
-
Assignee: (was: Sean McCarthy)

> dtest failure in 
> materialized_views_test.TestMaterializedViews.clustering_column_test
> -
>
> Key: CASSANDRA-11698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11698
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> recent failure, test has flapped before a while back.
> {noformat}
> Expecting 2 users, got 1
> {noformat}
> http://cassci.datastax.com/job/cassandra-3.0_dtest/688/testReport/materialized_views_test/TestMaterializedViews/clustering_column_test
> Failed on CassCI build cassandra-3.0_dtest #688



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11715:

Status: Ready to Commit  (was: Patch Available)

> Make GCInspector's MIN_LOG_DURATION configurable
> 
>
> Key: CASSANDRA-11715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11715
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>Assignee: Jeff Jirsa
>Priority: Minor
>  Labels: lhf
> Fix For: 2.2.8, 3.0.9, 3.9
>
>
> It's common for people to run C* with the G1 collector on appropriately-sized 
> heaps.  Quite often, the target pause time is set to 500ms, but GCI fires on 
> anything over 200ms.  We can already control the warn threshold, but these 
> are acceptable GCs for the configuration and create noise at the INFO log 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11715:

   Resolution: Fixed
Fix Version/s: 3.9
   3.0.9
   2.2.8
   Status: Resolved  (was: Ready to Commit)

[committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=f0d1d75ebf10beff6d24323c03c57e29dcd38c15]

> Make GCInspector's MIN_LOG_DURATION configurable
> 
>
> Key: CASSANDRA-11715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11715
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>Assignee: Jeff Jirsa
>Priority: Minor
>  Labels: lhf
> Fix For: 2.2.8, 3.0.9, 3.9
>
>
> It's common for people to run C* with the G1 collector on appropriately-sized 
> heaps.  Quite often, the target pause time is set to 500ms, but GCI fires on 
> anything over 200ms.  We can already control the warn threshold, but these 
> are acceptable GCs for the configuration and create noise at the INFO log 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-11446) dtest failure in scrub_test.TestScrub.test_nodetool_scrub

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-11446:
-
Comment: was deleted

(was: Filed a PR here: https://github.com/riptano/cassandra-dtest/pull/1086)

> dtest failure in scrub_test.TestScrub.test_nodetool_scrub
> -
>
> Key: CASSANDRA-11446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11446
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Jim Witschey
>  Labels: dtest
>
> test_nodetool_scrub is failing on trunk with offheap memtables. The failure 
> is in this assertion:
> {{self.assertEqual(initial_sstables, scrubbed_sstables)}}
> Example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/95/testReport/scrub_test/TestScrub/test_nodetool_scrub/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[10/10] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[jmckenzie]

Merge branch 'cassandra-3.9' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9fed0844
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9fed0844
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9fed0844

Branch: refs/heads/trunk
Commit: 9fed08449db4a446ac874d14178f86d20de97b18
Parents: 1f74142 76188e9
Author: Josh McKenzie 
Authored: Mon Jul 11 16:30:25 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:30:25 2016 -0400

--
 conf/cassandra.yaml   |  6 ++
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fed0844/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--

[01/10] cassandra git commit: Make GCInspector min log duration configurable

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 9a8406f2f -> f0d1d75eb
  refs/heads/cassandra-3.0 5861cd8fe -> e99ee1995
  refs/heads/cassandra-3.9 56abaca04 -> 76188e952
  refs/heads/trunk 1f74142d7 -> 9fed08449


Make GCInspector min log duration configurable

Patch by jjirsa; reviewed by jmckenzie for CASSANDRA-11715


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d1d75e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d1d75e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d1d75e

Branch: refs/heads/cassandra-2.2
Commit: f0d1d75ebf10beff6d24323c03c57e29dcd38c15
Parents: 9a8406f
Author: Jeff Jirsa 
Authored: Mon Jul 11 16:27:04 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:27:04 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 35e94d2..4ad798a 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -858,9 +858,14 @@ inter_dc_tcp_nodelay: false
 tracetype_query_ttl: 86400
 tracetype_repair_ttl: 604800
 
+# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+# This threshold can be adjusted to minimize logging if necessary
+# gc_log_threshold_in_ms: 200
+
 # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+# If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+# INFO level
 # Adjust the threshold based on your application throughput requirement
-# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 # gc_warn_threshold_in_ms: 1000
 
 # UDFs (user defined functions) are disabled by default.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index 9736a03..ede4560 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -254,6 +254,7 @@ public class Config
 public volatile Long index_summary_capacity_in_mb;
 public volatile int index_summary_resize_interval_in_minutes = 60;
 
+public int gc_log_threshold_in_ms = 200;
 public int gc_warn_threshold_in_ms = 0;
 
 private static final CsvPreference STANDARD_SURROUNDING_SPACES_NEED_QUOTES 
= new CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index d3a5028..f1acfc4 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -366,6 +366,11 @@ public class DatabaseDescriptor
 }
 paritionerName = partitioner.getClass().getCanonicalName();
 
+if (config.gc_log_threshold_in_ms < 0)
+{
+throw new ConfigurationException("gc_log_threshold_in_ms must be a 
positive integer");
+}
+
 if (conf.gc_warn_threshold_in_ms < 0)
 {
 throw new ConfigurationException("gc_warn_threshold_in_ms must be 
a positive integer");
@@ -1801,6 +1806,11 @@ public class DatabaseDescriptor
 return conf.windows_timer_interval;
 }
 
+public static long getGCLogThreshold()
+{
+return conf.gc_log_threshold_in_ms;
+}
+
 public static long getGCWarnThreshold()
 {
 return conf.gc_warn_threshold_in_ms;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/service/GCInspector.java
--
diff --git a/src/java/org/apache/cassandra/service/GCInspector.java 
b/src/java/org/apache/cassandra/service/GCInspector.java
index de5acc0..31de151 100644
--- a/src/java/org/apache/cassandra/service/GCInspector.java
+++ b/src/java/org/apache/cassandra/service/GCInspector.java
@@ -48,7 +48,7 @@ public class GCInspector implements 

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e99ee199
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e99ee199
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e99ee199

Branch: refs/heads/cassandra-3.0
Commit: e99ee19950e764ca55331d7c814e965bef359a4f
Parents: 5861cd8 f0d1d75
Author: Josh McKenzie 
Authored: Mon Jul 11 16:28:13 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:28:22 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/conf/cassandra.yaml
--
diff --cc conf/cassandra.yaml
index 4b92f64,4ad798a..09d2094
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@@ -924,21 -858,23 +924,26 @@@ inter_dc_tcp_nodelay: fals
  tracetype_query_ttl: 86400
  tracetype_repair_ttl: 604800
  
+ # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+ # This threshold can be adjusted to minimize logging if necessary
+ # gc_log_threshold_in_ms: 200
+ 
  # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+ # If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+ # INFO level
  # Adjust the threshold based on your application throughput requirement
- # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 -# gc_warn_threshold_in_ms: 1000
 +gc_warn_threshold_in_ms: 1000
  
  # UDFs (user defined functions) are disabled by default.
 -# As of Cassandra 2.2, there is no security manager or anything else in place 
that
 -# prevents execution of evil code. CASSANDRA-9402 will fix this issue for 
Cassandra 3.0.
 -# This will inherently be backwards-incompatible with any 2.2 UDF that 
perform insecure
 -# operations such as opening a socket or writing to the filesystem.
 +# As of Cassandra 3.0 there is a sandbox in place that should prevent 
execution of evil code.
  enable_user_defined_functions: false
  
 +# Enables scripted UDFs (JavaScript UDFs).
 +# Java UDFs are always enabled, if enable_user_defined_functions is true.
 +# Enable this option to be able to use UDFs with "language javascript" or any 
custom JSR-223 provider.
 +# This option has no effect, if enable_user_defined_functions is false.
 +enable_scripted_user_defined_functions: false
 +
  # The default Windows kernel timer and scheduling resolution is 15.6ms for 
power conservation.
  # Lowering this value on Windows can provide much tighter latency and better 
throughput, however
  # some virtualized environments may see a negative performance impact from 
changing this setting

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/src/java/org/apache/cassandra/config/Config.java
--
diff --cc src/java/org/apache/cassandra/config/Config.java
index b49e14c,ede4560..2bd23b5
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -265,8 -254,12 +265,9 @@@ public class Confi
  public volatile Long index_summary_capacity_in_mb;
  public volatile int index_summary_resize_interval_in_minutes = 60;
  
+ public int gc_log_threshold_in_ms = 200;
  public int gc_warn_threshold_in_ms = 0;
  
 -private static final CsvPreference 
STANDARD_SURROUNDING_SPACES_NEED_QUOTES = new 
CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)
 -  
.surroundingSpacesNeedQuotes(true).build();
 -
  // TTL for different types of trace events.
  public int tracetype_query_ttl = (int) TimeUnit.DAYS.toSeconds(1);
  public int tracetype_repair_ttl = (int) TimeUnit.DAYS.toSeconds(7);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 2083e42f,f1acfc4..100bcf4
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -1934,51 -1801,16 +1939,56 @@@ public class DatabaseDescripto
  return conf.enable_user_defined_functions;
  }
  
 -public static int getWindowsTimerInterval()
 +

[04/10] cassandra git commit: Make GCInspector min log duration configurable

[jmckenzie]

Make GCInspector min log duration configurable

Patch by jjirsa; reviewed by jmckenzie for CASSANDRA-11715


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d1d75e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d1d75e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d1d75e

Branch: refs/heads/trunk
Commit: f0d1d75ebf10beff6d24323c03c57e29dcd38c15
Parents: 9a8406f
Author: Jeff Jirsa 
Authored: Mon Jul 11 16:27:04 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:27:04 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 35e94d2..4ad798a 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -858,9 +858,14 @@ inter_dc_tcp_nodelay: false
 tracetype_query_ttl: 86400
 tracetype_repair_ttl: 604800
 
+# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+# This threshold can be adjusted to minimize logging if necessary
+# gc_log_threshold_in_ms: 200
+
 # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+# If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+# INFO level
 # Adjust the threshold based on your application throughput requirement
-# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 # gc_warn_threshold_in_ms: 1000
 
 # UDFs (user defined functions) are disabled by default.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index 9736a03..ede4560 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -254,6 +254,7 @@ public class Config
 public volatile Long index_summary_capacity_in_mb;
 public volatile int index_summary_resize_interval_in_minutes = 60;
 
+public int gc_log_threshold_in_ms = 200;
 public int gc_warn_threshold_in_ms = 0;
 
 private static final CsvPreference STANDARD_SURROUNDING_SPACES_NEED_QUOTES 
= new CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index d3a5028..f1acfc4 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -366,6 +366,11 @@ public class DatabaseDescriptor
 }
 paritionerName = partitioner.getClass().getCanonicalName();
 
+if (config.gc_log_threshold_in_ms < 0)
+{
+throw new ConfigurationException("gc_log_threshold_in_ms must be a 
positive integer");
+}
+
 if (conf.gc_warn_threshold_in_ms < 0)
 {
 throw new ConfigurationException("gc_warn_threshold_in_ms must be 
a positive integer");
@@ -1801,6 +1806,11 @@ public class DatabaseDescriptor
 return conf.windows_timer_interval;
 }
 
+public static long getGCLogThreshold()
+{
+return conf.gc_log_threshold_in_ms;
+}
+
 public static long getGCWarnThreshold()
 {
 return conf.gc_warn_threshold_in_ms;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/service/GCInspector.java
--
diff --git a/src/java/org/apache/cassandra/service/GCInspector.java 
b/src/java/org/apache/cassandra/service/GCInspector.java
index de5acc0..31de151 100644
--- a/src/java/org/apache/cassandra/service/GCInspector.java
+++ b/src/java/org/apache/cassandra/service/GCInspector.java
@@ -48,7 +48,7 @@ public class GCInspector implements NotificationListener, 
GCInspectorMXBean
 {
 public static final String MBEAN_NAME = 
"org.apache.cassandra.service:type=GCInspector";
 private static final Logger logger = 
LoggerFactory.getLogger(GCInspector.class);
-final static long 

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[jmckenzie]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76188e95
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76188e95
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76188e95

Branch: refs/heads/cassandra-3.9
Commit: 76188e9520d0aed8da287cffd80122e1069ddcae
Parents: 56abaca e99ee19
Author: Josh McKenzie 
Authored: Mon Jul 11 16:29:58 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:29:58 2016 -0400

--
 conf/cassandra.yaml   |  6 ++
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/76188e95/conf/cassandra.yaml
--
diff --cc conf/cassandra.yaml
index 076a729,09d2094..d79423e
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@@ -1051,6 -924,16 +1051,12 @@@ inter_dc_tcp_nodelay: fals
  tracetype_query_ttl: 86400
  tracetype_repair_ttl: 604800
  
+ # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+ # This threshold can be adjusted to minimize logging if necessary
+ # gc_log_threshold_in_ms: 200
+ 
 -# GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+ # If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+ # INFO level
 -# Adjust the threshold based on your application throughput requirement
 -gc_warn_threshold_in_ms: 1000
 -
  # UDFs (user defined functions) are disabled by default.
  # As of Cassandra 3.0 there is a sandbox in place that should prevent 
execution of evil code.
  enable_user_defined_functions: false

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76188e95/src/java/org/apache/cassandra/config/Config.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76188e95/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 1375a39,100bcf4..38dce11
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -2169,11 -1984,11 +2174,16 @@@ public class DatabaseDescripto
  conf.user_function_timeout_policy = userFunctionTimeoutPolicy;
  }
  
+ public static long getGCLogThreshold()
+ {
+ return conf.gc_log_threshold_in_ms;
+ }
+ 
 +public static EncryptionContext getEncryptionContext()
 +{
 +return encryptionContext;
 +}
 +
  public static long getGCWarnThreshold()
  {
  return conf.gc_warn_threshold_in_ms;

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[jmckenzie]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76188e95
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76188e95
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76188e95

Branch: refs/heads/trunk
Commit: 76188e9520d0aed8da287cffd80122e1069ddcae
Parents: 56abaca e99ee19
Author: Josh McKenzie 
Authored: Mon Jul 11 16:29:58 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:29:58 2016 -0400

--
 conf/cassandra.yaml   |  6 ++
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/76188e95/conf/cassandra.yaml
--
diff --cc conf/cassandra.yaml
index 076a729,09d2094..d79423e
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@@ -1051,6 -924,16 +1051,12 @@@ inter_dc_tcp_nodelay: fals
  tracetype_query_ttl: 86400
  tracetype_repair_ttl: 604800
  
+ # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+ # This threshold can be adjusted to minimize logging if necessary
+ # gc_log_threshold_in_ms: 200
+ 
 -# GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+ # If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+ # INFO level
 -# Adjust the threshold based on your application throughput requirement
 -gc_warn_threshold_in_ms: 1000
 -
  # UDFs (user defined functions) are disabled by default.
  # As of Cassandra 3.0 there is a sandbox in place that should prevent 
execution of evil code.
  enable_user_defined_functions: false

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76188e95/src/java/org/apache/cassandra/config/Config.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76188e95/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 1375a39,100bcf4..38dce11
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -2169,11 -1984,11 +2174,16 @@@ public class DatabaseDescripto
  conf.user_function_timeout_policy = userFunctionTimeoutPolicy;
  }
  
+ public static long getGCLogThreshold()
+ {
+ return conf.gc_log_threshold_in_ms;
+ }
+ 
 +public static EncryptionContext getEncryptionContext()
 +{
 +return encryptionContext;
 +}
 +
  public static long getGCWarnThreshold()
  {
  return conf.gc_warn_threshold_in_ms;

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e99ee199
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e99ee199
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e99ee199

Branch: refs/heads/trunk
Commit: e99ee19950e764ca55331d7c814e965bef359a4f
Parents: 5861cd8 f0d1d75
Author: Josh McKenzie 
Authored: Mon Jul 11 16:28:13 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:28:22 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/conf/cassandra.yaml
--
diff --cc conf/cassandra.yaml
index 4b92f64,4ad798a..09d2094
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@@ -924,21 -858,23 +924,26 @@@ inter_dc_tcp_nodelay: fals
  tracetype_query_ttl: 86400
  tracetype_repair_ttl: 604800
  
+ # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+ # This threshold can be adjusted to minimize logging if necessary
+ # gc_log_threshold_in_ms: 200
+ 
  # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+ # If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+ # INFO level
  # Adjust the threshold based on your application throughput requirement
- # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 -# gc_warn_threshold_in_ms: 1000
 +gc_warn_threshold_in_ms: 1000
  
  # UDFs (user defined functions) are disabled by default.
 -# As of Cassandra 2.2, there is no security manager or anything else in place 
that
 -# prevents execution of evil code. CASSANDRA-9402 will fix this issue for 
Cassandra 3.0.
 -# This will inherently be backwards-incompatible with any 2.2 UDF that 
perform insecure
 -# operations such as opening a socket or writing to the filesystem.
 +# As of Cassandra 3.0 there is a sandbox in place that should prevent 
execution of evil code.
  enable_user_defined_functions: false
  
 +# Enables scripted UDFs (JavaScript UDFs).
 +# Java UDFs are always enabled, if enable_user_defined_functions is true.
 +# Enable this option to be able to use UDFs with "language javascript" or any 
custom JSR-223 provider.
 +# This option has no effect, if enable_user_defined_functions is false.
 +enable_scripted_user_defined_functions: false
 +
  # The default Windows kernel timer and scheduling resolution is 15.6ms for 
power conservation.
  # Lowering this value on Windows can provide much tighter latency and better 
throughput, however
  # some virtualized environments may see a negative performance impact from 
changing this setting

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/src/java/org/apache/cassandra/config/Config.java
--
diff --cc src/java/org/apache/cassandra/config/Config.java
index b49e14c,ede4560..2bd23b5
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -265,8 -254,12 +265,9 @@@ public class Confi
  public volatile Long index_summary_capacity_in_mb;
  public volatile int index_summary_resize_interval_in_minutes = 60;
  
+ public int gc_log_threshold_in_ms = 200;
  public int gc_warn_threshold_in_ms = 0;
  
 -private static final CsvPreference 
STANDARD_SURROUNDING_SPACES_NEED_QUOTES = new 
CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)
 -  
.surroundingSpacesNeedQuotes(true).build();
 -
  // TTL for different types of trace events.
  public int tracetype_query_ttl = (int) TimeUnit.DAYS.toSeconds(1);
  public int tracetype_repair_ttl = (int) TimeUnit.DAYS.toSeconds(7);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 2083e42f,f1acfc4..100bcf4
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -1934,51 -1801,16 +1939,56 @@@ public class DatabaseDescripto
  return conf.enable_user_defined_functions;
  }
  
 -public static int getWindowsTimerInterval()
 +public 

[02/10] cassandra git commit: Make GCInspector min log duration configurable

[jmckenzie]

Make GCInspector min log duration configurable

Patch by jjirsa; reviewed by jmckenzie for CASSANDRA-11715


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d1d75e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d1d75e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d1d75e

Branch: refs/heads/cassandra-3.0
Commit: f0d1d75ebf10beff6d24323c03c57e29dcd38c15
Parents: 9a8406f
Author: Jeff Jirsa 
Authored: Mon Jul 11 16:27:04 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:27:04 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 35e94d2..4ad798a 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -858,9 +858,14 @@ inter_dc_tcp_nodelay: false
 tracetype_query_ttl: 86400
 tracetype_repair_ttl: 604800
 
+# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+# This threshold can be adjusted to minimize logging if necessary
+# gc_log_threshold_in_ms: 200
+
 # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+# If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+# INFO level
 # Adjust the threshold based on your application throughput requirement
-# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 # gc_warn_threshold_in_ms: 1000
 
 # UDFs (user defined functions) are disabled by default.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index 9736a03..ede4560 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -254,6 +254,7 @@ public class Config
 public volatile Long index_summary_capacity_in_mb;
 public volatile int index_summary_resize_interval_in_minutes = 60;
 
+public int gc_log_threshold_in_ms = 200;
 public int gc_warn_threshold_in_ms = 0;
 
 private static final CsvPreference STANDARD_SURROUNDING_SPACES_NEED_QUOTES 
= new CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index d3a5028..f1acfc4 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -366,6 +366,11 @@ public class DatabaseDescriptor
 }
 paritionerName = partitioner.getClass().getCanonicalName();
 
+if (config.gc_log_threshold_in_ms < 0)
+{
+throw new ConfigurationException("gc_log_threshold_in_ms must be a 
positive integer");
+}
+
 if (conf.gc_warn_threshold_in_ms < 0)
 {
 throw new ConfigurationException("gc_warn_threshold_in_ms must be 
a positive integer");
@@ -1801,6 +1806,11 @@ public class DatabaseDescriptor
 return conf.windows_timer_interval;
 }
 
+public static long getGCLogThreshold()
+{
+return conf.gc_log_threshold_in_ms;
+}
+
 public static long getGCWarnThreshold()
 {
 return conf.gc_warn_threshold_in_ms;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/service/GCInspector.java
--
diff --git a/src/java/org/apache/cassandra/service/GCInspector.java 
b/src/java/org/apache/cassandra/service/GCInspector.java
index de5acc0..31de151 100644
--- a/src/java/org/apache/cassandra/service/GCInspector.java
+++ b/src/java/org/apache/cassandra/service/GCInspector.java
@@ -48,7 +48,7 @@ public class GCInspector implements NotificationListener, 
GCInspectorMXBean
 {
 public static final String MBEAN_NAME = 
"org.apache.cassandra.service:type=GCInspector";
 private static final Logger logger = 
LoggerFactory.getLogger(GCInspector.class);
-final 

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e99ee199
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e99ee199
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e99ee199

Branch: refs/heads/cassandra-3.9
Commit: e99ee19950e764ca55331d7c814e965bef359a4f
Parents: 5861cd8 f0d1d75
Author: Josh McKenzie 
Authored: Mon Jul 11 16:28:13 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:28:22 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/conf/cassandra.yaml
--
diff --cc conf/cassandra.yaml
index 4b92f64,4ad798a..09d2094
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@@ -924,21 -858,23 +924,26 @@@ inter_dc_tcp_nodelay: fals
  tracetype_query_ttl: 86400
  tracetype_repair_ttl: 604800
  
+ # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+ # This threshold can be adjusted to minimize logging if necessary
+ # gc_log_threshold_in_ms: 200
+ 
  # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+ # If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+ # INFO level
  # Adjust the threshold based on your application throughput requirement
- # By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 -# gc_warn_threshold_in_ms: 1000
 +gc_warn_threshold_in_ms: 1000
  
  # UDFs (user defined functions) are disabled by default.
 -# As of Cassandra 2.2, there is no security manager or anything else in place 
that
 -# prevents execution of evil code. CASSANDRA-9402 will fix this issue for 
Cassandra 3.0.
 -# This will inherently be backwards-incompatible with any 2.2 UDF that 
perform insecure
 -# operations such as opening a socket or writing to the filesystem.
 +# As of Cassandra 3.0 there is a sandbox in place that should prevent 
execution of evil code.
  enable_user_defined_functions: false
  
 +# Enables scripted UDFs (JavaScript UDFs).
 +# Java UDFs are always enabled, if enable_user_defined_functions is true.
 +# Enable this option to be able to use UDFs with "language javascript" or any 
custom JSR-223 provider.
 +# This option has no effect, if enable_user_defined_functions is false.
 +enable_scripted_user_defined_functions: false
 +
  # The default Windows kernel timer and scheduling resolution is 15.6ms for 
power conservation.
  # Lowering this value on Windows can provide much tighter latency and better 
throughput, however
  # some virtualized environments may see a negative performance impact from 
changing this setting

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/src/java/org/apache/cassandra/config/Config.java
--
diff --cc src/java/org/apache/cassandra/config/Config.java
index b49e14c,ede4560..2bd23b5
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -265,8 -254,12 +265,9 @@@ public class Confi
  public volatile Long index_summary_capacity_in_mb;
  public volatile int index_summary_resize_interval_in_minutes = 60;
  
+ public int gc_log_threshold_in_ms = 200;
  public int gc_warn_threshold_in_ms = 0;
  
 -private static final CsvPreference 
STANDARD_SURROUNDING_SPACES_NEED_QUOTES = new 
CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)
 -  
.surroundingSpacesNeedQuotes(true).build();
 -
  // TTL for different types of trace events.
  public int tracetype_query_ttl = (int) TimeUnit.DAYS.toSeconds(1);
  public int tracetype_repair_ttl = (int) TimeUnit.DAYS.toSeconds(7);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e99ee199/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 2083e42f,f1acfc4..100bcf4
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -1934,51 -1801,16 +1939,56 @@@ public class DatabaseDescripto
  return conf.enable_user_defined_functions;
  }
  
 -public static int getWindowsTimerInterval()
 +

[03/10] cassandra git commit: Make GCInspector min log duration configurable

[jmckenzie]

Make GCInspector min log duration configurable

Patch by jjirsa; reviewed by jmckenzie for CASSANDRA-11715


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d1d75e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d1d75e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d1d75e

Branch: refs/heads/cassandra-3.9
Commit: f0d1d75ebf10beff6d24323c03c57e29dcd38c15
Parents: 9a8406f
Author: Jeff Jirsa 
Authored: Mon Jul 11 16:27:04 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 16:27:04 2016 -0400

--
 conf/cassandra.yaml   |  7 ++-
 src/java/org/apache/cassandra/config/Config.java  |  1 +
 .../org/apache/cassandra/config/DatabaseDescriptor.java   | 10 ++
 src/java/org/apache/cassandra/service/GCInspector.java|  2 +-
 4 files changed, 18 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 35e94d2..4ad798a 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -858,9 +858,14 @@ inter_dc_tcp_nodelay: false
 tracetype_query_ttl: 86400
 tracetype_repair_ttl: 604800
 
+# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
+# This threshold can be adjusted to minimize logging if necessary
+# gc_log_threshold_in_ms: 200
+
 # GC Pauses greater than gc_warn_threshold_in_ms will be logged at WARN level
+# If unset, all GC Pauses greater than gc_log_threshold_in_ms will log at
+# INFO level
 # Adjust the threshold based on your application throughput requirement
-# By default, Cassandra logs GC Pauses greater than 200 ms at INFO level
 # gc_warn_threshold_in_ms: 1000
 
 # UDFs (user defined functions) are disabled by default.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index 9736a03..ede4560 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -254,6 +254,7 @@ public class Config
 public volatile Long index_summary_capacity_in_mb;
 public volatile int index_summary_resize_interval_in_minutes = 60;
 
+public int gc_log_threshold_in_ms = 200;
 public int gc_warn_threshold_in_ms = 0;
 
 private static final CsvPreference STANDARD_SURROUNDING_SPACES_NEED_QUOTES 
= new CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index d3a5028..f1acfc4 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -366,6 +366,11 @@ public class DatabaseDescriptor
 }
 paritionerName = partitioner.getClass().getCanonicalName();
 
+if (config.gc_log_threshold_in_ms < 0)
+{
+throw new ConfigurationException("gc_log_threshold_in_ms must be a 
positive integer");
+}
+
 if (conf.gc_warn_threshold_in_ms < 0)
 {
 throw new ConfigurationException("gc_warn_threshold_in_ms must be 
a positive integer");
@@ -1801,6 +1806,11 @@ public class DatabaseDescriptor
 return conf.windows_timer_interval;
 }
 
+public static long getGCLogThreshold()
+{
+return conf.gc_log_threshold_in_ms;
+}
+
 public static long getGCWarnThreshold()
 {
 return conf.gc_warn_threshold_in_ms;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d1d75e/src/java/org/apache/cassandra/service/GCInspector.java
--
diff --git a/src/java/org/apache/cassandra/service/GCInspector.java 
b/src/java/org/apache/cassandra/service/GCInspector.java
index de5acc0..31de151 100644
--- a/src/java/org/apache/cassandra/service/GCInspector.java
+++ b/src/java/org/apache/cassandra/service/GCInspector.java
@@ -48,7 +48,7 @@ public class GCInspector implements NotificationListener, 
GCInspectorMXBean
 {
 public static final String MBEAN_NAME = 
"org.apache.cassandra.service:type=GCInspector";
 private static final Logger logger = 
LoggerFactory.getLogger(GCInspector.class);
-final 

[jira] [Commented] (CASSANDRA-11446) dtest failure in scrub_test.TestScrub.test_nodetool_scrub

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371574#comment-15371574
 ] 

Jim Witschey commented on CASSANDRA-11446:
--

Filed a PR here: https://github.com/riptano/cassandra-dtest/pull/1086

> dtest failure in scrub_test.TestScrub.test_nodetool_scrub
> -
>
> Key: CASSANDRA-11446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11446
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Jim Witschey
>  Labels: dtest
>
> test_nodetool_scrub is failing on trunk with offheap memtables. The failure 
> is in this assertion:
> {{self.assertEqual(initial_sstables, scrubbed_sstables)}}
> Example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/95/testReport/scrub_test/TestScrub/test_nodetool_scrub/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9667) strongly consistent membership and ownership

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-9667:
-
Reviewer: Jason Brown

> strongly consistent membership and ownership
> 
>
> Key: CASSANDRA-9667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9667
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jason Brown
>Assignee: Joel Knighton
>  Labels: LWT, membership, ownership
> Fix For: 3.x
>
>
> Currently, there is advice to users to "wait two minutes between adding new 
> nodes" in order for new node tokens, et al, to propagate. Further, as there's 
> no coordination amongst joining node wrt token selection, new nodes can end 
> up selecting ranges that overlap with other joining nodes. This causes a lot 
> of duplicate streaming from the existing source nodes as they shovel out the 
> bootstrap data for those new nodes.
> This ticket proposes creating a mechanism that allows strongly consistent 
> membership and ownership changes in cassandra such that changes are performed 
> in a linearizable and safe manner. The basic idea is to use LWT operations 
> over a global system table, and leverage the linearizability of LWT for 
> ensuring the safety of cluster membership/ownership state changes. This work 
> is inspired by Riak's claimant module.
> The existing workflows for node join, decommission, remove, replace, and 
> range move (there may be others I'm not thinking of) will need to be modified 
> to participate in this scheme, as well as changes to nodetool to enable them.
> Note: we distinguish between membership and ownership in the following ways: 
> for membership we mean "a host in this cluster and it's state". For 
> ownership, we mean "what tokens (or ranges) does each node own"; these nodes 
> must already be a member to be assigned tokens.
> A rough draft sketch of how the 'add new node' workflow might look like is: 
> new nodes would no longer create tokens themselves, but instead contact a 
> member of a Paxos cohort (via a seed). The cohort member will generate the 
> tokens and execute a LWT transaction, ensuring a linearizable change to the 
> membership/ownership state. The updated state will then be disseminated via 
> the existing gossip.
> As for joining specifically, I think we could support two modes: auto-mode 
> and manual-mode. Auto-mode is for adding a single new node per LWT operation, 
> and would require no operator intervention (much like today). In manual-mode, 
> however, multiple new nodes could (somehow) signal their their intent to join 
> to the cluster, but will wait until an operator executes a nodetool command 
> that will trigger the token generation and LWT operation for all pending new 
> nodes. This will allow us better range partitioning and will make the 
> bootstrap streaming more efficient as we won't have overlapping range 
> requests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-9667) strongly consistent membership and ownership

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton reassigned CASSANDRA-9667:


Assignee: Joel Knighton  (was: Jason Brown)

> strongly consistent membership and ownership
> 
>
> Key: CASSANDRA-9667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9667
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jason Brown
>Assignee: Joel Knighton
>  Labels: LWT, membership, ownership
> Fix For: 3.x
>
>
> Currently, there is advice to users to "wait two minutes between adding new 
> nodes" in order for new node tokens, et al, to propagate. Further, as there's 
> no coordination amongst joining node wrt token selection, new nodes can end 
> up selecting ranges that overlap with other joining nodes. This causes a lot 
> of duplicate streaming from the existing source nodes as they shovel out the 
> bootstrap data for those new nodes.
> This ticket proposes creating a mechanism that allows strongly consistent 
> membership and ownership changes in cassandra such that changes are performed 
> in a linearizable and safe manner. The basic idea is to use LWT operations 
> over a global system table, and leverage the linearizability of LWT for 
> ensuring the safety of cluster membership/ownership state changes. This work 
> is inspired by Riak's claimant module.
> The existing workflows for node join, decommission, remove, replace, and 
> range move (there may be others I'm not thinking of) will need to be modified 
> to participate in this scheme, as well as changes to nodetool to enable them.
> Note: we distinguish between membership and ownership in the following ways: 
> for membership we mean "a host in this cluster and it's state". For 
> ownership, we mean "what tokens (or ranges) does each node own"; these nodes 
> must already be a member to be assigned tokens.
> A rough draft sketch of how the 'add new node' workflow might look like is: 
> new nodes would no longer create tokens themselves, but instead contact a 
> member of a Paxos cohort (via a seed). The cohort member will generate the 
> tokens and execute a LWT transaction, ensuring a linearizable change to the 
> membership/ownership state. The updated state will then be disseminated via 
> the existing gossip.
> As for joining specifically, I think we could support two modes: auto-mode 
> and manual-mode. Auto-mode is for adding a single new node per LWT operation, 
> and would require no operator intervention (much like today). In manual-mode, 
> however, multiple new nodes could (somehow) signal their their intent to join 
> to the cluster, but will wait until an operator executes a nodetool command 
> that will trigger the token generation and LWT operation for all pending new 
> nodes. This will allow us better range partitioning and will make the 
> bootstrap streaming more efficient as we won't have overlapping range 
> requests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371559#comment-15371559
 ] 

Alex Petrov commented on CASSANDRA-12144:
-

I've written a patch for scrub as well. 

I've also ran upgrade tests from 2.2 SSTables to 3.0 both through 
{{sstableupgrade}} and through "broken" trunk + {{scrub}}. All paths show same 
results for a large sample of data. I'm re-running all tests plus upgrade tests 
now.

> Undeletable rows after upgrading from 2.2.4 to 3.0.7
> 
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is 
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
>  user_id| id | since| type
> ++--+--
>  116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367832#comment-15367832
 ] 

Alex Petrov edited comment on CASSANDRA-12144 at 7/11/16 8:07 PM:
--

{{2.x}} storage format doesn't guarantee that there'll be a single range 
tombstone, or that tombstones will be in the certain order relative to the 
cells. Under some circumstances (which I unfortunately could not reproduce), we 
were in the situation when we had multiple tombstones, followed by the row:

{code}
[
{"key": "1",
 "cells": [["12345:_","12345:!",,"t",], (*1)
   ["12345:_","12345:!",,"t",], (*2)
   ["12345:","",],
   ["12345:c1","xx",],
   ["12345:c2","yy",]]}
]
{code}

Which was resulting into two rows: one tombstone made from the {{(*1)}} and 
second one, live row made from the tombstone and cells following it (since 
delete time was that the cells should be live). 

This was resulting into the two rows in the new storage format after 
{{sstableupgrade}}. During the iteration, the first tombstone row was read out, 
although since second row was also read out and since the rest of merge 
iterators (superceeding delete might have been in memtable, or any other 
sstable) were exhausted, it was treated as a completely normal live row. 
Undeletable since all deletes would only affect the tombstone, whose clustering 
was matching. 

I've made a patch that captures this edge case.

|[3.0|https://github.com/ifesdjeen/cassandra/tree/12144-3.0] 
|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-3.0-testall/]
 
|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-3.0-dtest/]
 |[upgrade 
tests|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/upgrade_tests-all-12144-3.0/]|
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12144-trunk] 
|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-trunk-testall/]
 
|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-trunk-dtest/]
 |[upgrade 
tests|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/upgrade_tests-all-12144-trunk/]|

I'll run the CI and submit patch if it's successful (particularly interested in 
upgrade dtests). 

After talking to [~slebresne], we might have to also provide a fix for the 
scrub tool that'd detect and fix such cases.

Very big thanks to [~stanislav] for providing information required to track 
this issue down.


was (Author: ifesdjeen):
{{2.x}} storage format doesn't guarantee that there'll be a single range 
tombstone, or that tombstones will be in the certain order relative to the 
cells. Under some circumstances (which I unfortunately could not reproduce), we 
were in the situation when we had multiple tombstones, followed by the row:

{code}
[
{"key": "1",
 "cells": [["12345:_","12345:!",,"t",], (*1)
   ["12345:_","12345:!",,"t",], (*2)
   ["12345:","",],
   ["12345:c1","xx",],
   ["12345:c2","yy",]]}
]
{code}

Which was resulting into two rows: one tombstone made from the {{(*1)}} and 
second one, live row made from the tombstone and cells following it (since 
delete time was that the cells should be live). 

This was resulting into the two rows in the new storage format after 
{{sstableupgrade}}. During the iteration, the first tombstone row was read out, 
although since second row was also read out and since the rest of merge 
iterators (superceeding delete might have been in memtable, or any other 
sstable) were exhausted, it was treated as a completely normal live row. 
Undeletable since all deletes would only affect the tombstone, whose clustering 
was matching. 

I've made a patch that captures this edge case.

|[3.0|https://github.com/ifesdjeen/cassandra/tree/12144-3.0] 
|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-3.0-testall/]
 
|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-3.0-dtest/]
 |
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12144-trunk] 
|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-trunk-testall/]
 
|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12144-trunk-dtest/]
 |

I'll run the CI and submit patch if it's successful (particularly interested in 
upgrade dtests). 

After talking to [~slebresne], we might have to also provide a fix for the 
scrub tool that'd detect and fix such cases.

Very big thanks to [~stanislav] for providing information required to track 
this issue down.

> Undeletable rows after upgrading from 2.2.4 to 3.0.7
> 
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
>  Issue Type: Bug
>   

[jira] [Updated] (CASSANDRA-12171) counter mismatch during rolling upgrade from 2.2 to 3.0

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch updated CASSANDRA-12171:
---
Reproduced In: 3.0.x

> counter mismatch during rolling upgrade from 2.2 to 3.0
> ---
>
> Key: CASSANDRA-12171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12171
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Aleksey Yeschenko
>
> This may occur on other versions, but 3.0 is where I observed it recently.
> N=RF=3, counter writes at quorum, reads at quorum.
> This is being seen on some upgrade tests I'm currently repairing here: 
> https://github.com/riptano/cassandra-dtest/tree/upgrade_counters_fix (this 
> branch is to resolve an issue where counters were not being properly tested 
> during rolling upgrade tests).
> The test runs a continuous counter incrementing process, as well as a 
> continuous counter checking process. Once a counter value has been verified, 
> the test code makes it eligible to be incremented again.
> The test is encountering the problem when trying to check an expected counter 
> value and not matching expectations, for example:
> {noformat}
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in 
> _bootstrap
> self.run()
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
>   File 
> "/home/rhatch/git/cstar/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
>  line 210, in counter_checker
> tester.assertEqual(expected_count, actual_count)
>   File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
>   File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> AssertionError: 1 != 2
> ERROR
> {noformat}
> To check if something else could be going on, I did an experiment where I 
> changed the test to not upgrade nodes (just drain, stop, start) and the 
> mismatch didn't occur in several attempts. So it appears something about 
> upgrading is possibly the culprit.
> To run the test and repro locally:
> {noformat}
> grab my dtest branch at 
> https://github.com/riptano/cassandra-dtest/tree/upgrade_counters_fix
> export UPGRADE_TEST_RUN=true
> nosetests -v 
> upgrade_tests/upgrade_through_versions_test.py:TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_test
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12171) counter mismatch during rolling upgrade from 2.2 to 3.0

[Russ Hatch (JIRA)]

Russ Hatch created CASSANDRA-12171:
--

 Summary: counter mismatch during rolling upgrade from 2.2 to 3.0
 Key: CASSANDRA-12171
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12171
 Project: Cassandra
  Issue Type: Bug
Reporter: Russ Hatch
Assignee: Aleksey Yeschenko


This may occur on other versions, but 3.0 is where I observed it recently.

N=RF=3, counter writes at quorum, reads at quorum.

This is being seen on some upgrade tests I'm currently repairing here: 
https://github.com/riptano/cassandra-dtest/tree/upgrade_counters_fix (this 
branch is to resolve an issue where counters were not being properly tested 
during rolling upgrade tests).

The test runs a continuous counter incrementing process, as well as a 
continuous counter checking process. Once a counter value has been verified, 
the test code makes it eligible to be incremented again.

The test is encountering the problem when trying to check an expected counter 
value and not matching expectations, for example:
{noformat}
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
  File 
"/home/rhatch/git/cstar/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
 line 210, in counter_checker
tester.assertEqual(expected_count, actual_count)
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
raise self.failureException(msg)
AssertionError: 1 != 2
ERROR
{noformat}

To check if something else could be going on, I did an experiment where I 
changed the test to not upgrade nodes (just drain, stop, start) and the 
mismatch didn't occur in several attempts. So it appears something about 
upgrading is possibly the culprit.

To run the test and repro locally:
{noformat}
grab my dtest branch at 
https://github.com/riptano/cassandra-dtest/tree/upgrade_counters_fix

export UPGRADE_TEST_RUN=true

nosetests -v 
upgrade_tests/upgrade_through_versions_test.py:TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_test
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11575) Add out-of-process testing for CDC

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11575:

Reviewer:   (was: Joshua McKenzie)

> Add out-of-process testing for CDC
> --
>
> Key: CASSANDRA-11575
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11575
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Local Write-Read Paths
>Reporter: Carl Yeksigian
>Assignee: Joshua McKenzie
> Fix For: 3.x
>
> Attachments: 11575.tgz, 11575.tgz
>
>
> There are currently no dtests for the new cdc feature. We should have some, 
> at least to ensure that the cdc files have a lifecycle that makes sense, and 
> make sure that things like a continually cleaning daemon and a lazy daemon 
> have the properties we expect; for this, we don't need to actually process 
> the files, but make sure they fit the characteristics we expect from them. A 
> more complex daemon would need to be written in Java.
> I already hit a problem where if the cdc is over capacity, the cdc properly 
> throws the WTE, but it will not reset after the overflow directory is 
> undersize again. It is supposed to correct the size within 250ms and allow 
> more writes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11575) Add out-of-process testing for CDC

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11575:

Assignee: Joshua McKenzie  (was: Carl Yeksigian)
  Status: Open  (was: Patch Available)

> Add out-of-process testing for CDC
> --
>
> Key: CASSANDRA-11575
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11575
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Local Write-Read Paths
>Reporter: Carl Yeksigian
>Assignee: Joshua McKenzie
> Fix For: 3.x
>
> Attachments: 11575.tgz, 11575.tgz
>
>
> There are currently no dtests for the new cdc feature. We should have some, 
> at least to ensure that the cdc files have a lifecycle that makes sense, and 
> make sure that things like a continually cleaning daemon and a lazy daemon 
> have the properties we expect; for this, we don't need to actually process 
> the files, but make sure they fit the characteristics we expect from them. A 
> more complex daemon would need to be written in Java.
> I already hit a problem where if the cdc is over capacity, the cdc properly 
> throws the WTE, but it will not reset after the overflow directory is 
> undersize again. It is supposed to correct the size within 250ms and allow 
> more writes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11446) dtest failure in scrub_test.TestScrub.test_nodetool_scrub

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371455#comment-15371455
 ] 

Jim Witschey commented on CASSANDRA-11446:
--

Filed a PR here: https://github.com/riptano/cassandra-dtest/pull/1086


> dtest failure in scrub_test.TestScrub.test_nodetool_scrub
> -
>
> Key: CASSANDRA-11446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11446
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Jim Witschey
>  Labels: dtest
>
> test_nodetool_scrub is failing on trunk with offheap memtables. The failure 
> is in this assertion:
> {{self.assertEqual(initial_sstables, scrubbed_sstables)}}
> Example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/95/testReport/scrub_test/TestScrub/test_nodetool_scrub/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12018) CDC follow-ups

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12018:

Status: Ready to Commit  (was: Patch Available)

> CDC follow-ups
> --
>
> Key: CASSANDRA-12018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12018
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>
> h6. Platform independent implementation of DirectorySizeCalculator
> On linux, simplify to 
> {{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
> h6. Refactor DirectorySizeCalculator
> bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets, 
> the listFiles step? Either list the files and just loop through them, or do 
> the walkFileTree operation – you are now doing the same work twice. Use a 
> plain long instead of the atomic as the class is still thread-unsafe.
> h6. TolerateErrorsInSection should not depend on previous SyncSegment status 
> in CommitLogReader
> bq. tolerateErrorsInSection &=: I don't think it was intended for the value 
> to depend on previous iterations.
> h6. Refactor interface of SImpleCachedBufferPool
> bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int 
> size) which should automatically reallocate if the available size is less, 
> and not expose a setter at all.
> h6. Change CDC exception to WriteFailureException instead of 
> WriteTimeoutException
> h6. Remove unused CommitLogTest.testRecovery(byte[] logData)
> h6. NoSpamLogger a message when at CDC capacity



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12018) CDC follow-ups

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12018:

Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

Committed. Thanks for the review!

> CDC follow-ups
> --
>
> Key: CASSANDRA-12018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12018
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>
> h6. Platform independent implementation of DirectorySizeCalculator
> On linux, simplify to 
> {{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
> h6. Refactor DirectorySizeCalculator
> bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets, 
> the listFiles step? Either list the files and just loop through them, or do 
> the walkFileTree operation – you are now doing the same work twice. Use a 
> plain long instead of the atomic as the class is still thread-unsafe.
> h6. TolerateErrorsInSection should not depend on previous SyncSegment status 
> in CommitLogReader
> bq. tolerateErrorsInSection &=: I don't think it was intended for the value 
> to depend on previous iterations.
> h6. Refactor interface of SImpleCachedBufferPool
> bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int 
> size) which should automatically reallocate if the available size is less, 
> and not expose a setter at all.
> h6. Change CDC exception to WriteFailureException instead of 
> WriteTimeoutException
> h6. Remove unused CommitLogTest.testRecovery(byte[] logData)
> h6. NoSpamLogger a message when at CDC capacity



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: CDC Follow-ups

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9bf9ea740 -> 1f74142d7


CDC Follow-ups

Patch by jmckenzie; reviewed by blambov for CASSANDRA-12018


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1f74142d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1f74142d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1f74142d

Branch: refs/heads/trunk
Commit: 1f74142d756b3201acf0fe684943c972e7471782
Parents: 9bf9ea7
Author: Josh McKenzie 
Authored: Mon Jul 11 15:18:04 2016 -0400
Committer: Josh McKenzie 
Committed: Mon Jul 11 15:18:04 2016 -0400

--
 .../org/apache/cassandra/db/Directories.java| 11 +++--
 .../cassandra/db/commitlog/CommitLogReader.java |  3 +-
 .../commitlog/CommitLogSegmentManagerCDC.java   | 49 
 .../db/commitlog/CompressedSegment.java | 15 +-
 .../db/commitlog/EncryptedSegment.java  | 16 +++
 .../db/commitlog/SimpleCachedBufferPool.java| 17 +--
 .../utils/DirectorySizeCalculator.java  | 39 ++--
 .../test/microbench/DirectorySizerBench.java|  1 -
 .../cassandra/db/commitlog/CommitLogTest.java   | 13 --
 9 files changed, 62 insertions(+), 102 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1f74142d/src/java/org/apache/cassandra/db/Directories.java
--
diff --git a/src/java/org/apache/cassandra/db/Directories.java 
b/src/java/org/apache/cassandra/db/Directories.java
index 2a55992..87527e8 100644
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@ -920,7 +920,7 @@ public class Directories
 if (!input.isDirectory())
 return 0;
 
-SSTableSizeSummer visitor = new 
SSTableSizeSummer(sstableLister(Directories.OnTxnErr.THROW).listFiles());
+SSTableSizeSummer visitor = new SSTableSizeSummer(input, 
sstableLister(Directories.OnTxnErr.THROW).listFiles());
 try
 {
 Files.walkFileTree(input.toPath(), visitor);
@@ -1006,9 +1006,11 @@ public class Directories
 
 private class SSTableSizeSummer extends DirectorySizeCalculator
 {
-SSTableSizeSummer(List files)
+private final HashSet toSkip;
+SSTableSizeSummer(File path, List files)
 {
-super(files);
+super(path);
+toSkip = new HashSet<>(files);
 }
 
 @Override
@@ -1019,8 +1021,7 @@ public class Directories
 return pair != null
 && pair.left.ksname.equals(metadata.ksName)
 && pair.left.cfname.equals(metadata.cfName)
-&& !visited.contains(fileName)
-&& !alive.contains(fileName);
+&& !toSkip.contains(fileName);
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1f74142d/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
index 4594080..c797482 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
@@ -190,7 +190,8 @@ public class CommitLogReader
 ReadStatusTracker statusTracker = new 
ReadStatusTracker(mutationLimit, tolerateTruncation);
 for (CommitLogSegmentReader.SyncSegment syncSegment : 
segmentReader)
 {
-statusTracker.tolerateErrorsInSection &= 
syncSegment.toleratesErrorsInSection;
+// Only tolerate truncation if we allow in both global and 
segment
+statusTracker.tolerateErrorsInSection = tolerateTruncation 
& syncSegment.toleratesErrorsInSection;
 
 // Skip segments that are completely behind the desired 
minPosition
 if (desc.id == minPosition.segmentId && 
syncSegment.endPosition < minPosition.position)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1f74142d/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerCDC.java
--
diff --git 
a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerCDC.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerCDC.java
index 15944bd..1fac735 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerCDC.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerCDC.java
@@ -20,9 +20,11 @@ 

[jira] [Resolved] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie resolved CASSANDRA-12168.

Resolution: Cannot Reproduce

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>
> With a C* 2.1 node querying a table with DCT columns from a 3.0 node we see 
> the following exception:
> {code}
> java.lang.IllegalArgumentException: null
>   at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:611) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.DynamicCompositeType.getComparator(DynamicCompositeType.java:97)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.DynamicCompositeType.getComparator(DynamicCompositeType.java:118)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compareCustom(AbstractCompositeType.java:63)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:157) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ClusteringComparator.compareComponent(ClusteringComparator.java:166)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:137)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at org.apache.cassandra.db.Slices$Builder.add(Slices.java:206) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher.filterIfStale(KeysSearcher.java:193)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher.access$400(KeysSearcher.java:38)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher$1.prepareNext(KeysSearcher.java:107)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher$1.hasNext(KeysSearcher.java:72)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:295)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:289) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:47)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371439#comment-15371439
 ] 

Anthony Cozzie commented on CASSANDRA-12168:


It seems a) my analysis was incorrect because I missed that the C* code was 
checking != vs = and b) this got fixed in 3.0.8 somewhere.  Marking as 'cannot 
reproduce'.  At least we are on our way to 6 digit jira numbers . . .

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>
> With a C* 2.1 node querying a table with DCT columns from a 3.0 node we see 
> the following exception:
> {code}
> java.lang.IllegalArgumentException: null
>   at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:611) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.DynamicCompositeType.getComparator(DynamicCompositeType.java:97)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.DynamicCompositeType.getComparator(DynamicCompositeType.java:118)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compareCustom(AbstractCompositeType.java:63)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:157) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ClusteringComparator.compareComponent(ClusteringComparator.java:166)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:137)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at org.apache.cassandra.db.Slices$Builder.add(Slices.java:206) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher.filterIfStale(KeysSearcher.java:193)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher.access$400(KeysSearcher.java:38)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher$1.prepareNext(KeysSearcher.java:107)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.index.internal.keys.KeysSearcher$1.hasNext(KeysSearcher.java:72)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:295)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:289) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:47)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
> {code}



--
This 

[jira] [Commented] (CASSANDRA-12149) NullPointerException on SELECT with SASI index

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371429#comment-15371429
 ] 

DOAN DuyHai commented on CASSANDRA-12149:
-

bq. How could I partition SELECT results hitting a single large Cassandra 
partition, when I do not know values of clustering columns?

 Use server-side paging:

 "SELECT * FROM mytable WHERE partition = 'xxx'"

 Then set a fetchSize on the statement using the driver and then retrieve an 
Iterator from the ResultSet and start iterating with 
{{while(iterator.hasNext())}}

> NullPointerException on SELECT with SASI index
> --
>
> Key: CASSANDRA-12149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12149
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Andrey Konstantinov
> Attachments: CASSANDRA-12149.txt
>
>
> If I execute the sequence of queries (see the attached file), Cassandra 
> aborts a connection reporting NPE on server side. SELECT query without token 
> range filter works, but does not work when token range filter is specified. 
> My intent was to issue multiple SELECT queries targeting the same single 
> partition, filtered by a column indexed by SASI, partitioning results by 
> different token ranges.
> Output from cqlsh on SELECT is the following:
> cqlsh> SELECT namespace, entity, timestamp, feature1, feature2 FROM 
> mykeyspace.myrecordtable WHERE namespace = 'ns2' AND entity = 'entity2' AND 
> feature1 > 11 AND feature1 < 31  AND token(namespace, entity) <= 
> 9223372036854775807;
> ServerError:  message="java.lang.NullPointerException">



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12170) Create ci job to run ant test-cdc weekly

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371356#comment-15371356
 ] 

Michael Shuler commented on CASSANDRA-12170:


This is running on ASF's Jenkins.
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-test-cdc/

> Create ci job to run ant test-cdc weekly
> 
>
> Key: CASSANDRA-12170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12170
> Project: Cassandra
>  Issue Type: Test
>Reporter: Joshua McKenzie
>Assignee: Michael Shuler
>Priority: Minor
>
> With CASSANDRA-8844, we added a new ant target 'test-cdc'. This exercises all 
> unit tests w/a CDC-enabled CommitLogSegmentManager. We need a CI job to run 
> this weekly and catch any potential regressions in functionality.
> Should be able to copy from 'testall' on trunk and just change the target. 
> Only needed on trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12170) Create ci job to run ant test-cdc weekly

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-12170.

Resolution: Done

> Create ci job to run ant test-cdc weekly
> 
>
> Key: CASSANDRA-12170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12170
> Project: Cassandra
>  Issue Type: Test
>Reporter: Joshua McKenzie
>Assignee: Michael Shuler
>Priority: Minor
>
> With CASSANDRA-8844, we added a new ant target 'test-cdc'. This exercises all 
> unit tests w/a CDC-enabled CommitLogSegmentManager. We need a CI job to run 
> this weekly and catch any potential regressions in functionality.
> Should be able to copy from 'testall' on trunk and just change the target. 
> Only needed on trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12170) Create ci job to run ant test-cdc weekly

[Joshua McKenzie (JIRA)]

Joshua McKenzie created CASSANDRA-12170:
---

 Summary: Create ci job to run ant test-cdc weekly
 Key: CASSANDRA-12170
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12170
 Project: Cassandra
  Issue Type: Test
Reporter: Joshua McKenzie
Assignee: Michael Shuler
Priority: Minor


With CASSANDRA-8844, we added a new ant target 'test-cdc'. This exercises all 
unit tests w/a CDC-enabled CommitLogSegmentManager. We need a CI job to run 
this weekly and catch any potential regressions in functionality.

Should be able to copy from 'testall' on trunk and just change the target. Only 
needed on trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12169) Create ci job to run ant test-cdc weekly

[Joshua McKenzie (JIRA)]

Joshua McKenzie created CASSANDRA-12169:
---

 Summary: Create ci job to run ant test-cdc weekly
 Key: CASSANDRA-12169
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12169
 Project: Cassandra
  Issue Type: Test
Reporter: Joshua McKenzie
Assignee: Michael Shuler
Priority: Minor


With CASSANDRA-8844, we added a new ant target 'test-cdc'. This exercises all 
unit tests w/a CDC-enabled CommitLogSegmentManager. We need a CI job to run 
this weekly and catch any potential regressions in functionality.

Should be able to copy from 'testall' on trunk and just change the target. Only 
needed on trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12155) proposeCallback.java is too spammy for debug.log

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12155:

Reviewer: Joshua McKenzie

> proposeCallback.java is too spammy for debug.log
> 
>
> Key: CASSANDRA-12155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12155
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Wei Deng
>Assignee: Wei Deng
>Priority: Minor
>
> As stated in [this wiki 
> page|https://wiki.apache.org/cassandra/LoggingGuidelines] derived from the 
> work on CASSANDRA-10241, the DEBUG level logging in debug.log is intended for 
> "+low frequency state changes or message passing. Non-critical path logs on 
> operation details, performance measurements or general troubleshooting 
> information.+"
> However, it appears that in a production deployment of C* 3.x, the LWT 
> message passing from ProposeCallback.java gets printed every 1-2 seconds, 
> which overwhelms debug.log from presenting the other important DEBUG level 
> logging messages, like the following:
> {noformat}
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:23:57,800  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:00,803  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:00,804  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:03,807  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:03,807  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:06,811  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:06,811  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:09,815  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:09,815  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:12,819  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:12,819  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:15,823  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:15,823  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:18,827  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:18,827  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:21,831  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:21,831  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:24,835  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:24,835  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:27,839  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:27,839  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:30,843  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:30,843  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:33,847  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:33,847  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:36,851  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:36,852  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:39,855  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG [SharedPool-Worker-2] 2016-07-09 05:24:39,855  ProposeCallback.java:62 
> - Propose response true from /10.240.0.3
> DEBUG [SharedPool-Worker-1] 2016-07-09 05:24:42,859  ProposeCallback.java:62 
> - Propose response true from /10.240.0.2
> DEBUG 

[jira] [Updated] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11715:

Reviewer: Joshua McKenzie

> Make GCInspector's MIN_LOG_DURATION configurable
> 
>
> Key: CASSANDRA-11715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11715
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>Assignee: Jeff Jirsa
>Priority: Minor
>  Labels: lhf
>
> It's common for people to run C* with the G1 collector on appropriately-sized 
> heaps.  Quite often, the target pause time is set to 500ms, but GCI fires on 
> anything over 200ms.  We can already control the warn threshold, but these 
> are acceptable GCs for the configuration and create noise at the INFO log 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11424:

Reviewer: Sylvain Lebresne

> Add support to "unset" JSON fields in prepared statements
> -
>
> Key: CASSANDRA-11424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11424
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ralf Steppacher
>Assignee: Oded Peer
>  Labels: client-impacting, cql
> Fix For: 3.8
>
> Attachments: 11424-trunk-V1.txt, 11424-trunk-V2.txt, 
> 11424-trunk-V3.txt
>
>
> CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and 
> {{UNSET}} prepared statement parameters.
> When inserting JSON objects it is not possible to profit from this as a 
> prepared statement only has one parameter that is bound to the JSON object as 
> a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for 
> columns omitted from the JSON object.
> Please extend on CASSANDRA-7304 to include JSON support.
> {color:grey}
> (My personal requirement is to be able to insert JSON objects with optional 
> fields without incurring the overhead of creating a tombstone of every column 
> not covered by the JSON object upon initial(!) insert.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8457) nio MessagingService

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8457:
---
Reviewer: Sylvain Lebresne

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11668) InterruptedException in HintsDispatcher

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11668:

Assignee: Aleksey Yeschenko

> InterruptedException in HintsDispatcher
> ---
>
> Key: CASSANDRA-11668
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11668
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Aleksey Yeschenko
>  Labels: dtest
>
> This exception was seen when trying to repro a test problem. The original 
> issue test problem appears to be a non-issue, but the exception seems like it 
> could be worth investigation.
> This happened on upgrade from 3.2.1 to 3.3 HEAD (a soon to be retired 
> test-case).
> The test does a rolling upgrade where nodes are one by one stopped, upgraded, 
> and started on the new version.
> The exception occurred some time after starting node1 on the upgraded 
> version, and upgrading/starting node2 on the new version. Node2 logged the 
> exception.
> {noformat}
> node2: ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_51]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> Caused by: java.lang.InterruptedException: null
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.checkInterrupted(WaitQueue.java:313)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUntil(WaitQueue.java:301)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.SimpleCondition.await(SimpleCondition.java:63)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:201)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   ... 11 common frames omitted
> Unexpected error in node2 log, error: 
> ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 CassandraDaemon.java:195 - 
> Exception in thread Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> 

[jira] [Commented] (CASSANDRA-11668) InterruptedException in HintsDispatcher

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371213#comment-15371213
 ] 

Russ Hatch commented on CASSANDRA-11668:


If it's not appeared on any other tests (I'm not certain) I think we're ok to 
close this.

> InterruptedException in HintsDispatcher
> ---
>
> Key: CASSANDRA-11668
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11668
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>  Labels: dtest
>
> This exception was seen when trying to repro a test problem. The original 
> issue test problem appears to be a non-issue, but the exception seems like it 
> could be worth investigation.
> This happened on upgrade from 3.2.1 to 3.3 HEAD (a soon to be retired 
> test-case).
> The test does a rolling upgrade where nodes are one by one stopped, upgraded, 
> and started on the new version.
> The exception occurred some time after starting node1 on the upgraded 
> version, and upgrading/starting node2 on the new version. Node2 logged the 
> exception.
> {noformat}
> node2: ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_51]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> Caused by: java.lang.InterruptedException: null
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.checkInterrupted(WaitQueue.java:313)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUntil(WaitQueue.java:301)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.SimpleCondition.await(SimpleCondition.java:63)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:201)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   ... 11 common frames omitted
> Unexpected error in node2 log, error: 
> ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 CassandraDaemon.java:195 - 
> Exception in thread Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at 

[jira] [Updated] (CASSANDRA-11668) InterruptedException in HintsDispatcher

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch updated CASSANDRA-11668:
---
Assignee: (was: Russ Hatch)

> InterruptedException in HintsDispatcher
> ---
>
> Key: CASSANDRA-11668
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11668
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>  Labels: dtest
>
> This exception was seen when trying to repro a test problem. The original 
> issue test problem appears to be a non-issue, but the exception seems like it 
> could be worth investigation.
> This happened on upgrade from 3.2.1 to 3.3 HEAD (a soon to be retired 
> test-case).
> The test does a rolling upgrade where nodes are one by one stopped, upgraded, 
> and started on the new version.
> The exception occurred some time after starting node1 on the upgraded 
> version, and upgrading/starting node2 on the new version. Node2 logged the 
> exception.
> {noformat}
> node2: ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_51]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> Caused by: java.lang.InterruptedException: null
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.checkInterrupted(WaitQueue.java:313)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUntil(WaitQueue.java:301)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.SimpleCondition.await(SimpleCondition.java:63)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:201)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   ... 11 common frames omitted
> Unexpected error in node2 log, error: 
> ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 CassandraDaemon.java:195 - 
> Exception in thread Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> 

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-12168:

Description: 
With a C* 2.1 node querying a table with DCT columns from a 3.0 node we see the 
following exception:

{code}
java.lang.IllegalArgumentException: null
at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_66]
at 
org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:611) 
~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.marshal.DynamicCompositeType.getComparator(DynamicCompositeType.java:97)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.marshal.DynamicCompositeType.getComparator(DynamicCompositeType.java:118)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.compareCustom(AbstractCompositeType.java:63)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:157) 
~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ClusteringComparator.compareComponent(ClusteringComparator.java:166)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:137)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at org.apache.cassandra.db.Slices$Builder.add(Slices.java:206) 
~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.index.internal.keys.KeysSearcher.filterIfStale(KeysSearcher.java:193)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.index.internal.keys.KeysSearcher.access$400(KeysSearcher.java:38)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.index.internal.keys.KeysSearcher$1.prepareNext(KeysSearcher.java:107)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.index.internal.keys.KeysSearcher$1.hasNext(KeysSearcher.java:72)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:295)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:289) 
~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:47)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_66]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
 ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
 [cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
{code}

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>
> With a C* 2.1 node querying a table with DCT columns from a 3.0 node we see 
> the following exception:
> {code}
> java.lang.IllegalArgumentException: null
>   at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:611) 
> ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159]
>   

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-12168:

Fix Version/s: (was: 3.0.9)
   (was: 3.9)
   3.x
   3.0.x

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-12168:
---
Status: Open  (was: Patch Available)

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371172#comment-15371172
 ] 

Anthony Cozzie commented on CASSANDRA-12168:


Hmm, I may have been too hasty here, taking another look. 

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11978) StreamReader fails to write sstable if CF directory is symlink

[Arindam Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371171#comment-15371171
 ] 

Arindam Gupta commented on CASSANDRA-11978:
---

Ok thanks, just wondering if this can be reproducible using CCM? Currently I do 
not have any real cluster running to reproduce it.

> StreamReader fails to write sstable if CF directory is symlink
> --
>
> Key: CASSANDRA-11978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Michael Frisch
>  Labels: lhf
>
> I'm using Cassandra v2.2.6.  If the CF is stored as a symlink in the keyspace 
> directory on disk then StreamReader.createWriter fails because 
> Descriptor.fromFilename is passed the actual path on disk instead of path 
> with the symlink.
> Example:
> /path/to/data/dir/Keyspace/CFName -> /path/to/data/dir/AnotherDisk/CFName
> Descriptor.fromFilename is passed "/path/to/data/dir/AnotherDisk/CFName" 
> instead of "/path/to/data/dir/Keyspace/CFName", then it concludes that the 
> keyspace name is "AnotherDisk" which is erroneous. I've temporarily worked 
> around this by using cfs.keyspace.getName() to get the keyspace name and 
> cfs.name to get the CF name as those are correct.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-12168:
---
Labels: easyfix  (was: )

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-12168:
---
Component/s: Streaming and Messaging

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
>  Labels: easyfix
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-12168:
---
Fix Version/s: 3.9
   3.0.9

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371150#comment-15371150
 ] 

Anthony Cozzie commented on CASSANDRA-12168:


I'm guessing this code was never exercised due to CASSANDRA-12147, since the 
fix is really trivial

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-12168:
---
Status: Patch Available  (was: Open)

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie updated CASSANDRA-12168:
---
Attachment: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch

> DCT deserialization code incorrect in 3.0
> -
>
> Key: CASSANDRA-12168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anthony Cozzie
>Assignee: Anthony Cozzie
> Attachments: 0001-CASSANDRA-12168-fix-thrift-DCT-deserialization.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12168) DCT deserialization code incorrect in 3.0

[Anthony Cozzie (JIRA)]

Anthony Cozzie created CASSANDRA-12168:
--

 Summary: DCT deserialization code incorrect in 3.0
 Key: CASSANDRA-12168
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12168
 Project: Cassandra
  Issue Type: Bug
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10202) simplify CommitLogSegmentManager

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371097#comment-15371097
 ] 

Joshua McKenzie commented on CASSANDRA-10202:
-

We discussed running the testall target weekly with test-cdc. I'll follow up 
with the TE folks to ensure that gets scheduled.

> simplify CommitLogSegmentManager
> 
>
> Key: CASSANDRA-10202
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10202
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Jonathan Ellis
>Assignee: Branimir Lambov
>Priority: Minor
>
> Now that we only keep one active segment around we can simplify this from the 
> old recycling design.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10845) jmxmetrics_test.TestJMXMetrics.begin_test is failing

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey resolved CASSANDRA-10845.
--
Resolution: Incomplete

My PR deleting this test has been merged. Since this started with a patch, I've 
closed this and opened a new ticket:

https://issues.apache.org/jira/browse/CASSANDRA-12167

> jmxmetrics_test.TestJMXMetrics.begin_test is failing
> 
>
> Key: CASSANDRA-10845
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10845
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: Philip Thompson
>Assignee: DS Test Eng
>Priority: Minor
>  Labels: dtest
>
> This test is failing on 2.1-head. There appear to be structural issues with 
> the test, and no C* bug to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12167) Review JMX metrics coverage

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-12167:


 Summary: Review JMX metrics coverage
 Key: CASSANDRA-12167
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12167
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
Assignee: DS Test Eng


I just deleted the dtest that was meant to smoke test JMX metrics:

https://github.com/riptano/cassandra-dtest/pull/1085

The idea was that you'd read JMX metrics, run stress, then make sure the 
metrics went up, down, or stayed the same, as appropriate. This kind of 
coverage would be good to have.

I don't think we have it anywhere in the dtests, and it probably isn't 
appropriate in unit tests. We should check there's no coverage in the unit 
tests, and add some coverage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370533#comment-15370533
 ] 

Sergio Bossa edited comment on CASSANDRA-9318 at 7/11/16 4:06 PM:
--

[~Stefania],

bq. Do we track the case where we receive a failed response? Specifically, in 
ResponseVerbHandler.doVerb, shouldn't we call updateBackPressureState() also 
when the message is a failure response?

Good point, I focused more on the success case, due to dropped mutations, but 
that sounds like a good thing to do.

bq. If we receive a response after it has timed out, won't we count that 
request twice, incorrectly increasing the rate for that window?

But can that really happen? {{ResponseVerbHandler}} returns _before_ 
incrementing back-pressure if the callback is null (i.e. expired), and 
{{OutboundTcpConnection}} doesn't even send outbound messages if they're timed 
out, or am I missing something?

bq. I also argue that it is quite easy to comment out the strategy and to have 
an empty strategy in the code that means no backpressure.

Again, I believe this would make enabling/disabling back-pressure via JMX less 
user friendly.

bq. I think what we may need is a new companion snitch that sorts the replica 
by backpressure ratio

I do not think sorting replicas is what we really need, as you have to send the 
mutation to all replicas anyway. I think what you rather need is a way to 
pre-emptively fail if the write consistency level is not met by enough 
"non-overloaded" replicas, i.e.:
* If CL.ONE, fail if *all* replicas are overloaded.
* If CL.QUORUM, fail if *quorum* replicas are overloaded.
* if CL.ALL, fail if *any* replica is overloaded.

This can be easily accomplished in {{StorageProxy#sendToHintedEndpoints}}.

bq. the exception needs to be different. native_protocol_v4.spec clearly states

I missed that too :(

This leaves us with two options:
* Adding a new exception to the native protocol.
* Reusing a different exception, with {{WriteFailureException}} and 
{{UnavailableException}} the most likely candidates.

I'm currently leaning towards the latter option.

bq. By "load shedding by the replica" do we mean dropping mutations that have 
timed out or something else?

Yes.

bq. Regardless, there is the problem of ensuring that all nodes have 
backpressure enabled, which may not be trivial.

We only need to ensure the coordinator for that specific mutation has 
back-pressure enabled, and we could do this by "marking" the {{MessageOut}} 
with a special parameter, what do you think?


was (Author: sbtourist):
[~Stefania],

bq. Do we track the case where we receive a failed response? Specifically, in 
ResponseVerbHandler.doVerb, shouldn't we call updateBackPressureState() also 
when the message is a failure response?

Good point, I focused more on the success case, due to dropped mutations, but 
that sounds like a good thing to do.

bq. If we receive a response after it has timed out, won't we count that 
request twice, incorrectly increasing the rate for that window?

But can that really happen? {{ResponseVerbHandler}} returns _before_ 
incrementing back-pressure if the callback is null (i.e. expired), and 
{{OutboundTcpConnection}} doesn't even send outbound messages if they're timed 
out, or am I missing something?

bq. I also argue that it is quite easy to comment out the strategy and to have 
an empty strategy in the code that means no backpressure.

Again, I believe this would make enabling/disabling back-pressure via JMX less 
user friendly.

bq. I think what we may need is a new companion snitch that sorts the replica 
by backpressure ratio

I do not think sorting replicas is what we really need, as you have to send the 
mutation to all replicas anyway. I think what you rather need is a way to 
pre-emptively fail if the write consistency level is not met by enough 
"non-overloaded" replicas, i.e.:
* If CL.ONE, fail in *all* replicas are overloaded.
* If CL.QUORUM, fail if *quorum* replicas are overloaded.
* if CL.ALL, fail if *any* replica is overloaded.

This can be easily accomplished in {{StorageProxy#sendToHintedEndpoints}}.

bq. the exception needs to be different. native_protocol_v4.spec clearly states

I missed that too :(

This leaves us with two options:
* Adding a new exception to the native protocol.
* Reusing a different exception, with {{WriteFailureException}} and 
{{UnavailableException}} the most likely candidates.

I'm currently leaning towards the latter option.

bq. By "load shedding by the replica" do we mean dropping mutations that have 
timed out or something else?

Yes.

bq. Regardless, there is the problem of ensuring that all nodes have 
backpressure enabled, which may not be trivial.

We only need to ensure the coordinator for that specific mutation has 
back-pressure enabled, and we could do this by "marking" the {{MessageOut}} 
with a special parameter, what do 

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371063#comment-15371063
 ] 

Sergio Bossa commented on CASSANDRA-9318:
-

bq. One thing that worries me is, how do you distinguish between “node X is 
slow because we are writing too fast and we need to throttle clients down” and 
“node X is slow because it is dying, we need to ignore it and accept writes 
based on other replicas?”
I.e. this seems to implicitly push everyone to a kind of CL.ALL model once your 
threshold triggers, where if one replica is slow then we can't make progress.

This was already noted by [~slebresne], and you're both right, this initial 
implementation is heavily biased towards my specific use case :)
But, the above proposed solution should fix it:

bq. I think what you rather need is a way to pre-emptively fail if the write 
consistency level is not met by enough "non-overloaded" replicas, i.e.: If 
CL.ONE, fail if all replicas are overloaded...

Also, the exception would be sent to the client only if the low threshold is 
met, and only the first time it is met, for the duration of the back-pressure 
window (write RPC timeout), i.e.:
* Threshold is 0.1, outgoing requests are 100, incoming responses are 10, ratio 
is 0.1.
* Exception is thrown by all write requests for the current back-pressure 
window.
* The outgoing rate limiter is set at 10, which means the next ratio 
calculation will approach the sustainable rate, and even if replicas will still 
lag behind, the ratio will not go down to 0.1 _unless_ the incoming rate 
dramatically goes down to 1.

This is to say the chances of getting a meltdown due to "overloaded exceptions" 
is moderate (also, clients are supposed to adjust themselves when getting such 
exception), and the above proposal should make things play nicely with the CL 
too.

If you all agree with that, I'll move forward and make that change.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
> Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, 
> limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10845) jmxmetrics_test.TestJMXMetrics.begin_test is failing

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371064#comment-15371064
 ] 

Jim Witschey commented on CASSANDRA-10845:
--

I've opened this PR to delete that entire test file:

https://github.com/riptano/cassandra-dtest/pull/1085

If it's merged, I'll change this ticket to "smoke test jmxmetrics", it'd be 
nice to have that coverage.

> jmxmetrics_test.TestJMXMetrics.begin_test is failing
> 
>
> Key: CASSANDRA-10845
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10845
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: Philip Thompson
>Assignee: DS Test Eng
>Priority: Minor
>  Labels: dtest
>
> This test is failing on 2.1-head. There appear to be structural issues with 
> the test, and no C* bug to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371063#comment-15371063
 ] 

Sergio Bossa edited comment on CASSANDRA-9318 at 7/11/16 4:05 PM:
--

bq. One thing that worries me is, how do you distinguish between “node X is 
slow because we are writing too fast and we need to throttle clients down” and 
“node X is slow because it is dying, we need to ignore it and accept writes 
based on other replicas?” I.e. this seems to implicitly push everyone to a kind 
of CL.ALL model once your threshold triggers, where if one replica is slow then 
we can't make progress.

This was already noted by [~slebresne], and you're both right, this initial 
implementation is heavily biased towards my specific use case :)
But, the above proposed solution should fix it:

bq. I think what you rather need is a way to pre-emptively fail if the write 
consistency level is not met by enough "non-overloaded" replicas, i.e.: If 
CL.ONE, fail if all replicas are overloaded...

Also, the exception would be sent to the client only if the low threshold is 
met, and only the first time it is met, for the duration of the back-pressure 
window (write RPC timeout), i.e.:
* Threshold is 0.1, outgoing requests are 100, incoming responses are 10, ratio 
is 0.1.
* Exception is thrown by all write requests for the current back-pressure 
window.
* The outgoing rate limiter is set at 10, which means the next ratio 
calculation will approach the sustainable rate, and even if replicas will still 
lag behind, the ratio will not go down to 0.1 _unless_ the incoming rate 
dramatically goes down to 1.

This is to say the chances of getting a meltdown due to "overloaded exceptions" 
is moderate (also, clients are supposed to adjust themselves when getting such 
exception), and the above proposal should make things play nicely with the CL 
too.

If you all agree with that, I'll move forward and make that change.


was (Author: sbtourist):
bq. One thing that worries me is, how do you distinguish between “node X is 
slow because we are writing too fast and we need to throttle clients down” and 
“node X is slow because it is dying, we need to ignore it and accept writes 
based on other replicas?”
I.e. this seems to implicitly push everyone to a kind of CL.ALL model once your 
threshold triggers, where if one replica is slow then we can't make progress.

This was already noted by [~slebresne], and you're both right, this initial 
implementation is heavily biased towards my specific use case :)
But, the above proposed solution should fix it:

bq. I think what you rather need is a way to pre-emptively fail if the write 
consistency level is not met by enough "non-overloaded" replicas, i.e.: If 
CL.ONE, fail if all replicas are overloaded...

Also, the exception would be sent to the client only if the low threshold is 
met, and only the first time it is met, for the duration of the back-pressure 
window (write RPC timeout), i.e.:
* Threshold is 0.1, outgoing requests are 100, incoming responses are 10, ratio 
is 0.1.
* Exception is thrown by all write requests for the current back-pressure 
window.
* The outgoing rate limiter is set at 10, which means the next ratio 
calculation will approach the sustainable rate, and even if replicas will still 
lag behind, the ratio will not go down to 0.1 _unless_ the incoming rate 
dramatically goes down to 1.

This is to say the chances of getting a meltdown due to "overloaded exceptions" 
is moderate (also, clients are supposed to adjust themselves when getting such 
exception), and the above proposal should make things play nicely with the CL 
too.

If you all agree with that, I'll move forward and make that change.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
> Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, 
> limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12158) dtest failure in thrift_tests.TestMutations.test_describe_keyspace

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371048#comment-15371048
 ] 

Philip Thompson commented on CASSANDRA-12158:
-

Oh, that is most definitely the problem and needed fix.

> dtest failure in thrift_tests.TestMutations.test_describe_keyspace
> --
>
> Key: CASSANDRA-12158
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12158
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/492/testReport/thrift_tests/TestMutations/test_describe_keyspace
> Failed on CassCI build cassandra-2.1_dtest #492
> {code}
> Stacktrace
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/thrift_tests.py", line 1507, in 
> test_describe_keyspace
> assert len(kspaces) == 4, [x.name for x in kspaces]  # ['Keyspace2', 
> 'Keyspace1', 'system', 'system_traces']
> AssertionError: ['Keyspace2', 'system', 'Keyspace1', 'ValidKsForUpdate', 
> 'system_traces']
> {code}
> Related failures:
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/304/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/cassandra-3.0_dtest/767/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/264/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/trunk_dtest/1301/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/trunk_novnode_dtest/421/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/cassandra-3.9_dtest/6/testReport/thrift_tests/TestMutations/test_describe_keyspace/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12158) dtest failure in thrift_tests.TestMutations.test_describe_keyspace

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371042#comment-15371042
 ] 

Joel Knighton commented on CASSANDRA-12158:
---

This is almost certainly because {{thrift_tests.py}} moved to the 
ReusableClusterTest with [PR 
1065|https://github.com/riptano/cassandra-dtest/pull/1065]. Flakiness depends 
on test order execution, since the extra keyspaces' existence depends on order 
of test method execution. We could drop any potential extra keyspaces at the 
beginning of {{test_describe_keyspace}}.

> dtest failure in thrift_tests.TestMutations.test_describe_keyspace
> --
>
> Key: CASSANDRA-12158
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12158
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/492/testReport/thrift_tests/TestMutations/test_describe_keyspace
> Failed on CassCI build cassandra-2.1_dtest #492
> {code}
> Stacktrace
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/thrift_tests.py", line 1507, in 
> test_describe_keyspace
> assert len(kspaces) == 4, [x.name for x in kspaces]  # ['Keyspace2', 
> 'Keyspace1', 'system', 'system_traces']
> AssertionError: ['Keyspace2', 'system', 'Keyspace1', 'ValidKsForUpdate', 
> 'system_traces']
> {code}
> Related failures:
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/304/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/cassandra-3.0_dtest/767/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/264/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/trunk_dtest/1301/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/trunk_novnode_dtest/421/testReport/thrift_tests/TestMutations/test_describe_keyspace/
> http://cassci.datastax.com/job/cassandra-3.9_dtest/6/testReport/thrift_tests/TestMutations/test_describe_keyspace/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8876) Allow easier init script reuse

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-8876:
--
Status: Testing  (was: Patch Available)

> Allow easier init script reuse
> --
>
> Key: CASSANDRA-8876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Marko Asplund
>Assignee: Michael Shuler
> Fix For: 3.x
>
> Attachments: trunk-CASSANDRA-8876.txt
>
>
> Make it possible to reuse the Cassandra debian init script with different 
> configuration and Cassandra home paths by making paths configurable via 
> environment variables set in /etc/default/cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371010#comment-15371010
 ] 

Jonathan Ellis commented on CASSANDRA-9318:
---

If I understand this approach correctly, you're looking at the ratio of sent to 
acknowledged writes per replica, and throwing Unavailable if that gets too low 
for a given replica.  Very clever.

One thing that worries me is, how do you distinguish between “node X is slow 
because we are writing too fast and we need to throttle clients down” and “node 
X is slow because it is dying, we need to ignore it and accept writes based on 
other replicas?”

I.e. this seems to implicitly push everyone to a kind of CL.ALL model once your 
threshold triggers, where if one replica is slow then we can't make progress.

If we take a simpler approach of just bounding total outstanding requests to 
all replicas per coordinator, then we can avoid overload meltdown while 
allowing CL to continue to work as designed and tolerate as many slow replicas 
as the requested CL permits.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
> Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, 
> limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7961) Run dtests on ARM

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-7961.
---
Resolution: Later

Closing this out as Later to clean up queue. It's interesting, but I don't 
think I've seen any ARM users.

> Run dtests on ARM
> -
>
> Key: CASSANDRA-7961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7961
> Project: Cassandra
>  Issue Type: Test
>Reporter: Ryan McGuire
>Assignee: Michael Shuler
>Priority: Minor
>
> It looks pretty easy to setup an [emulated ARM 
> environment|https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap], 
> we might want to look into setting this up on Cassci.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9320) test-burn target should be run occasionally

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371028#comment-15371028
 ] 

Michael Shuler commented on CASSANDRA-9320:
---

Been attempting this on ASF's Jenkins
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-test-burn/

> test-burn target should be run occasionally
> ---
>
> Key: CASSANDRA-9320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9320
> Project: Cassandra
>  Issue Type: Test
>Reporter: Ariel Weisberg
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 3.x
>
>
> The tests are all concurrency tests right now so they need to run on the 
> largest  # of cores we have available. The tests are not configured to run 
> very long right now, but the intent is that they run for longer periods (days 
> even).
> They aren't described as high value right now because the code under test 
> hasn't change since first introduced so we can defer setting this job up 
> until higher priority things are done.
> I think we should still run them at some low frequency so they don't rot or 
> some change doesn't sneak in that effects them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11668) InterruptedException in HintsDispatcher

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371027#comment-15371027
 ] 

Jim Witschey commented on CASSANDRA-11668:
--

This on an upgrade path we don't test anymore:

bq. This happened on upgrade from 3.2.1 to 3.3 HEAD (a soon to be retired 
test-case).

Does that mean we don't care about it? Have we seen it since?

> InterruptedException in HintsDispatcher
> ---
>
> Key: CASSANDRA-11668
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11668
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Russ Hatch
>  Labels: dtest
>
> This exception was seen when trying to repro a test problem. The original 
> issue test problem appears to be a non-issue, but the exception seems like it 
> could be worth investigation.
> This happened on upgrade from 3.2.1 to 3.3 HEAD (a soon to be retired 
> test-case).
> The test does a rolling upgrade where nodes are one by one stopped, upgraded, 
> and started on the new version.
> The exception occurred some time after starting node1 on the upgraded 
> version, and upgrading/starting node2 on the new version. Node2 logged the 
> exception.
> {noformat}
> node2: ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_51]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> Caused by: java.lang.InterruptedException: null
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.checkInterrupted(WaitQueue.java:313)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUntil(WaitQueue.java:301)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.utils.concurrent.SimpleCondition.await(SimpleCondition.java:63)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:201)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   ... 11 common frames omitted
> Unexpected error in node2 log, error: 
> ERROR [HintsDispatcher:2] 2016-05-09 23:37:45,816 CassandraDaemon.java:195 - 
> Exception in thread Thread[HintsDispatcher:2,1,main]
> java.lang.AssertionError: java.lang.InterruptedException
>   at 
> org.apache.cassandra.hints.HintsDispatcher$Callback.await(HintsDispatcher.java:205)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:146)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:121) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:93) 
> ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:247)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
>  ~[apache-cassandra-3.2.1.jar:3.2.1]
>   at 

[jira] [Commented] (CASSANDRA-10845) jmxmetrics_test.TestJMXMetrics.begin_test is failing

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371020#comment-15371020
 ] 

Jim Witschey commented on CASSANDRA-10845:
--

Yeah, this test seems to be super, _super_ incorrect. But, I imagine there are 
higher-priority test failures to deal with? I've bumped this down to {{Minor}}.

> jmxmetrics_test.TestJMXMetrics.begin_test is failing
> 
>
> Key: CASSANDRA-10845
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10845
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: Philip Thompson
>Assignee: DS Test Eng
>Priority: Minor
>  Labels: dtest
>
> This test is failing on 2.1-head. There appear to be structural issues with 
> the test, and no C* bug to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10845) jmxmetrics_test.TestJMXMetrics.begin_test is failing

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10845:
-
Priority: Minor  (was: Major)

> jmxmetrics_test.TestJMXMetrics.begin_test is failing
> 
>
> Key: CASSANDRA-10845
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10845
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: Philip Thompson
>Assignee: DS Test Eng
>Priority: Minor
>  Labels: dtest
>
> This test is failing on 2.1-head. There appear to be structural issues with 
> the test, and no C* bug to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10915) netstats_test dtest fails on Windows, flaps on Linux

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371003#comment-15371003
 ] 

Jim Witschey commented on CASSANDRA-10915:
--

This hasn't failed on Linux in over 2 months (Jenkins's entire history), at 
least on the linked job. In light of that, I'm going to just label this with 
{{windows}}.

> netstats_test dtest fails on Windows, flaps on Linux
> 
>
> Key: CASSANDRA-10915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10915
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest, windows
> Fix For: 3.0.x
>
>
> jmx_test.py:TestJMX.netstats_test started failing hard on Windows about a 
> month ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/140/testReport/junit/jmx_test/TestJMX/netstats_test/history/?start=25
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/156/testReport/jmx_test/TestJMX/netstats_test/history/
> It fails when it is unable to connect to a node via JMX. I don't know if this 
> problem has any relationship to CASSANDRA-10913.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10915) netstats_test dtest fails on Windows, flaps on Linux

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10915:
-
Labels: dtest windows  (was: dtest)

> netstats_test dtest fails on Windows, flaps on Linux
> 
>
> Key: CASSANDRA-10915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10915
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest, windows
> Fix For: 3.0.x
>
>
> jmx_test.py:TestJMX.netstats_test started failing hard on Windows about a 
> month ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/140/testReport/junit/jmx_test/TestJMX/netstats_test/history/?start=25
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/156/testReport/jmx_test/TestJMX/netstats_test/history/
> It fails when it is unable to connect to a node via JMX. I don't know if this 
> problem has any relationship to CASSANDRA-10913.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11906) Unstable JVM due too many files when anticompacting big LCS tables

[Stefano Ortolani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370986#comment-15370986
 ] 

Stefano Ortolani commented on CASSANDRA-11906:
--

Small update: updated to 3.0.8, trying sequential repairs once again. No 
exception yet, but I see the number of open files being around 25K (still far 
from 100K for now)

> Unstable JVM due too many files when anticompacting big LCS tables
> --
>
> Key: CASSANDRA-11906
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11906
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefano Ortolani
>
> I have recently moved from C* 2.1.x to C* 3.0.6. The setup is quite 
> heavy:
>   - 13 nodes with spinning disks
>   - ~120 GB of data per node
>   - 50% of CFs are LCS and have quite wide rows.
>   - 2/3 CFs with LCS have more than 200 SStables
> Incremental repairs do not seem to play really well with that.
> I have been running some tests node by node by using the -pr option:
> {code:xml}
> nodetool -h localhost repair -pr keyscheme
> {code}
> and to my surprise the whole process takes quite some time (1 hour
> minimum, 8 hours if I haven't been repairing for 5/6 days).
> Yesterday I tried to run the command with the -seq option so to 
> decrease the number of simultanoues compactions. After a while
> the node on which I was running the repair simply died during
> the anticompaction phase with the following
> exception in the logs.
> {code:xml}
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-25 21:54:21,868 
> ScheduledReporter.java:119 - RuntimeException thrown from 
> GraphiteReporter#report. Exception was suppressed.
> java.lang.RuntimeException: Failed to list files in 
> /data/cassandra/data/keyschema/columnfamily-3996ce80b7ac11e48a9b6776bf484396
>   at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:57)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:691)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories$SSTableLister.listFiles(Directories.java:662)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories$TrueFilesSizeVisitor.(Directories.java:981)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories.getTrueAllocatedSizeIn(Directories.java:893)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories.trueSnapshotsSize(Directories.java:883) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.trueSnapshotsSize(ColumnFamilyStore.java:2332)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:637) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:634) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_91]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: 
> /data/cassandra/data/keyschema/columnfamily-3996ce80b7ac11e48a9b6776bf484396/ma_txn_anticompactionafterrepair_f20b50d0-22bd-11e6-970f-6f22464f4624.log:
>  Too many open files
>   at org.apache.cassandra.io.util.FileUtils.readLines(FileUtils.java:622) 
> 

[jira] [Updated] (CASSANDRA-10915) netstats_test dtest flaps

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10915:
-
Summary: netstats_test dtest flaps  (was: netstats_test dtest fails on 
Windows)

> netstats_test dtest flaps
> -
>
> Key: CASSANDRA-10915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10915
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>  Labels: dtest
> Fix For: 3.0.x
>
>
> jmx_test.py:TestJMX.netstats_test started failing hard on Windows about a 
> month ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/140/testReport/junit/jmx_test/TestJMX/netstats_test/history/?start=25
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/156/testReport/jmx_test/TestJMX/netstats_test/history/
> It fails when it is unable to connect to a node via JMX. I don't know if this 
> problem has any relationship to CASSANDRA-10913.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10915) netstats_test dtest flaps

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10915:
-
Assignee: DS Test Eng

> netstats_test dtest flaps
> -
>
> Key: CASSANDRA-10915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10915
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.0.x
>
>
> jmx_test.py:TestJMX.netstats_test started failing hard on Windows about a 
> month ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/140/testReport/junit/jmx_test/TestJMX/netstats_test/history/?start=25
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/156/testReport/jmx_test/TestJMX/netstats_test/history/
> It fails when it is unable to connect to a node via JMX. I don't know if this 
> problem has any relationship to CASSANDRA-10913.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10915) netstats_test dtest fails on Windows, flaps on Linux

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10915:
-
Summary: netstats_test dtest fails on Windows, flaps on Linux  (was: 
netstats_test dtest flaps)

> netstats_test dtest fails on Windows, flaps on Linux
> 
>
> Key: CASSANDRA-10915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10915
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.0.x
>
>
> jmx_test.py:TestJMX.netstats_test started failing hard on Windows about a 
> month ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/140/testReport/junit/jmx_test/TestJMX/netstats_test/history/?start=25
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/156/testReport/jmx_test/TestJMX/netstats_test/history/
> It fails when it is unable to connect to a node via JMX. I don't know if this 
> problem has any relationship to CASSANDRA-10913.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11290) dtest failure in materialized_views_test.TestMaterializedViewsConsistency.single_partition_consistent_reads_after_write_test

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370953#comment-15370953
 ] 

Jim Witschey commented on CASSANDRA-11290:
--

[~carlyeks] Any more recent thoughts on this? Should we block 3.9 on it?

> dtest failure in 
> materialized_views_test.TestMaterializedViewsConsistency.single_partition_consistent_reads_after_write_test
> 
>
> Key: CASSANDRA-11290
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11290
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Carl Yeksigian
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/26/testReport/materialized_views_test/TestMaterializedViewsConsistency/single_partition_consistent_reads_after_write_test
> Failed on CassCI build trunk_offheap_dtest #26
> Failing infrequently but same error on at least two of those infrequent flaps:
> {noformat}
> Connection to 127.0.0.2 was closed
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11403) Serializer/Version mismatch during upgrades to C* 3.0

[Anthony Cozzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370947#comment-15370947
 ] 

Anthony Cozzie commented on CASSANDRA-11403:


I skimmed the code for CASSANDRA-11393, and it seems to be doing exactly what I 
was planning to do, i.e. using a single serializer that forwards requests to 
3.0 or legacy serializers.  That should fix the race condition.  I went ahead 
and marked this as a duplicate.

> Serializer/Version mismatch during upgrades to C* 3.0
> -
>
> Key: CASSANDRA-11403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11403
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>
> The problem line seems to be:
> {code}
> MessageOut message = 
> readCommand.createMessage(MessagingService.instance().getVersion(endpoint));
> {code}
> SinglePartitionReadCommand then picks the serializer based on the version:
> {code}
> return new MessageOut<>(MessagingService.Verb.READ, this, version < 
> MessagingService.VERSION_30 ? legacyReadCommandSerializer : serializer);
> {code}
> However, OutboundTcpConnectionPool will test the payload size vs the version 
> from its smallMessages connection:
> {code}
> return msg.payloadSize(smallMessages.getTargetVersion()) > 
> LARGE_MESSAGE_THRESHOLD
> {code}
> Which is set when the connection/pool is created:
> {code}
> targetVersion = MessagingService.instance().getVersion(pool.endPoint());
> {code}
> During an upgrade, this state can change between these two calls leading the 
> 3.0 serializer being used on 2.x packets and the following stacktrace:
> ERROR [OptionalTasks:1] 2016-03-07 19:53:06,445  CassandraDaemon.java:195 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:632)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:536)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor$NeverSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:214)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:918)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:251)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:77)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) 
> 

[jira] [Resolved] (CASSANDRA-11403) Serializer/Version mismatch during upgrades to C* 3.0

[Anthony Cozzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Cozzie resolved CASSANDRA-11403.

Resolution: Duplicate

> Serializer/Version mismatch during upgrades to C* 3.0
> -
>
> Key: CASSANDRA-11403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11403
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>
> The problem line seems to be:
> {code}
> MessageOut message = 
> readCommand.createMessage(MessagingService.instance().getVersion(endpoint));
> {code}
> SinglePartitionReadCommand then picks the serializer based on the version:
> {code}
> return new MessageOut<>(MessagingService.Verb.READ, this, version < 
> MessagingService.VERSION_30 ? legacyReadCommandSerializer : serializer);
> {code}
> However, OutboundTcpConnectionPool will test the payload size vs the version 
> from its smallMessages connection:
> {code}
> return msg.payloadSize(smallMessages.getTargetVersion()) > 
> LARGE_MESSAGE_THRESHOLD
> {code}
> Which is set when the connection/pool is created:
> {code}
> targetVersion = MessagingService.instance().getVersion(pool.endPoint());
> {code}
> During an upgrade, this state can change between these two calls leading the 
> 3.0 serializer being used on 2.x packets and the following stacktrace:
> ERROR [OptionalTasks:1] 2016-03-07 19:53:06,445  CassandraDaemon.java:195 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:632)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:536)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor$NeverSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:214)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:918)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:251)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:77)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252) 
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247) 
> 

[jira] [Resolved] (CASSANDRA-12157) Cassandra solr_query not working after upgrading to DSE 5

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson resolved CASSANDRA-12157.
-
Resolution: Invalid

This jira is for issues with the open source Apache Cassandra project, which 
does not have the solr functionality you are referring.

> Cassandra solr_query not working after upgrading to DSE 5
> -
>
> Key: CASSANDRA-12157
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12157
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Sreeraju.V
>
> After upgrading to DSE 5 solr_query is not working. Below is the new DSE, 
> cqlsh and Cassandra versions.
> [cqlsh 5.0.1 | Cassandra 3.0.7.1158 | DSE 5.0.0 | CQL spec 3.4.0 | Native 
> protocol v4]
> I am connecting using PHP Driver. The exception catching is
> Must not send frame with CUSTOM_PAYLOAD flag for native protocol version 
> < 4
> and the
> error code is 33554442
> When I run the same query on cqlsh it is working but not through the 
> Php-driver.
> $countSearchParam = '{"q":"'.$searchParam.'" }';
> try{
> $countStatement = $this->session->prepare(
> "SELECT count(*) FROM table WHERE solr_query = ? ");
> $countresults = $this->session->execute($countStatement, new 
> Cassandra\ExecutionOptions(array(
> 'arguments' => array($countSearchParam)
> )));
> foreach ($countresults as $row) {
> $cntArr = get_object_vars($row['count']);
> $totCount = $cntArr['value'];
> }
> }catch(Exception $e){



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12157) Cassandra solr_query not working after upgrading to DSE 5

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370890#comment-15370890
 ] 

Jeremiah Jordan commented on CASSANDRA-12157:
-

This is a problem with DSE not open source Apache Cassandra.
It is a known problem. 
http://docs.datastax.com/en/latest-dse/datastax_enterprise/RNdse.html?scroll=RNdse__501KnownIss

> Cassandra solr_query not working after upgrading to DSE 5
> -
>
> Key: CASSANDRA-12157
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12157
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Sreeraju.V
>
> After upgrading to DSE 5 solr_query is not working. Below is the new DSE, 
> cqlsh and Cassandra versions.
> [cqlsh 5.0.1 | Cassandra 3.0.7.1158 | DSE 5.0.0 | CQL spec 3.4.0 | Native 
> protocol v4]
> I am connecting using PHP Driver. The exception catching is
> Must not send frame with CUSTOM_PAYLOAD flag for native protocol version 
> < 4
> and the
> error code is 33554442
> When I run the same query on cqlsh it is working but not through the 
> Php-driver.
> $countSearchParam = '{"q":"'.$searchParam.'" }';
> try{
> $countStatement = $this->session->prepare(
> "SELECT count(*) FROM table WHERE solr_query = ? ");
> $countresults = $this->session->execute($countStatement, new 
> Cassandra\ExecutionOptions(array(
> 'arguments' => array($countSearchParam)
> )));
> foreach ($countresults as $row) {
> $cntArr = get_object_vars($row['count']);
> $totCount = $cntArr['value'];
> }
> }catch(Exception $e){



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12105) ThriftServer.stop is not thread safe

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12105:
--
Priority: Minor  (was: Critical)

> ThriftServer.stop is not thread safe
> 
>
> Key: CASSANDRA-12105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12105
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Brian Wawok
>Assignee: Brian Wawok
>Priority: Minor
> Fix For: 2.2.8, 3.0.9, 3.9
>
> Attachments: patch1.txt, patch2.txt
>
>
> There is a small thread safety issue in ThriftServer.stop(). If we have 
> multiple calls to stop, one thread may NPE or otherwise do bad stuff.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12105) ThriftServer.stop is not thread safe

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370827#comment-15370827
 ] 

Aleksey Yeschenko commented on CASSANDRA-12105:
---

Committed version 2 to 2.2 and merged upwards. Not going to touch 2.1, as this 
is not a critical bug, by any definition. Also, in the future, please submit 
properly formatted patches that can be applied directly, as per [this 
document|https://docs.google.com/document/d/1d_AzYQo74de9utbbpyXxW2w-b0__sFhC7b24ibiJqjw/view].
 Thank you.

> ThriftServer.stop is not thread safe
> 
>
> Key: CASSANDRA-12105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12105
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Brian Wawok
>Assignee: Brian Wawok
>Priority: Critical
> Fix For: 2.2.8, 3.0.9, 3.9
>
> Attachments: patch1.txt, patch2.txt
>
>
> There is a small thread safety issue in ThriftServer.stop(). If we have 
> multiple calls to stop, one thread may NPE or otherwise do bad stuff.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12105) ThriftServer.stop is not thread safe

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12105:
--
   Resolution: Fixed
Fix Version/s: (was: 2.1.x)
   3.9
   3.0.9
   2.2.8
   Status: Resolved  (was: Patch Available)

> ThriftServer.stop is not thread safe
> 
>
> Key: CASSANDRA-12105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12105
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Brian Wawok
>Assignee: Brian Wawok
>Priority: Critical
> Fix For: 2.2.8, 3.0.9, 3.9
>
> Attachments: patch1.txt, patch2.txt
>
>
> There is a small thread safety issue in ThriftServer.stop(). If we have 
> multiple calls to stop, one thread may NPE or otherwise do bad stuff.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10884) test_refresh_schema_on_timeout_error dtest flapping on CassCI

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370824#comment-15370824
 ] 

Jim Witschey commented on CASSANDRA-10884:
--

Attempting repro here:

http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/166/

> test_refresh_schema_on_timeout_error dtest flapping on CassCI
> -
>
> Key: CASSANDRA-10884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10884
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.0.x
>
>
> These tests create keyspaces and tables through cqlsh, then runs {{DESCRIBE}} 
> to confirm they were successfully created. These tests flap under the novnode 
> dtest runs:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_refresh_schema_on_timeout_error/history/
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_refresh_schema_on_timeout_error/history/
> I have not reproduced this locally on Linux.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[aleksey]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/56abaca0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/56abaca0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/56abaca0

Branch: refs/heads/trunk
Commit: 56abaca0466411739895523d0c3a81a7630ab9f0
Parents: 371a147 5861cd8
Author: Aleksey Yeschenko 
Authored: Mon Jul 11 15:00:33 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Mon Jul 11 15:01:04 2016 +0100

--
 CHANGES.txt| 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/56abaca0/CHANGES.txt
--
diff --cc CHANGES.txt
index f65b1f4,70210a8..da8216f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,4 +1,9 @@@
 -3.0.9
 +3.9
 + * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
 + * Fix hdr logging for single operation workloads (CASSANDRA-12145)
 + * Fix SASI PREFIX search in CONTAINS mode with partial terms 
(CASSANDRA-12073)
 + * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 +Merged from 3.0:
- 3.0.9
   * Fix migration of static thrift column names with non-text comparators 
(CASSANDRA-12147)
   * Fix upgrading sparse tables that are incorrectly marked as dense 
(CASSANDRA-11315)
   * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)