[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832752#comment-15832752
 ] 

Jeff Jirsa commented on CASSANDRA-11163:


It persists until all sstables are rewritten - the bloom filter is recalculated 
on startup but not persisted to disk until compaction runs. You can force the 
bloom filter to be rewritten without waiting dircompaction or this bug to be 
fixed by running "nodetool upgradesstables -a" to rewrite all of the sstables 
(note that this can be IO intensive and impact your latencies)

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
> Fix For: 3.x
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 

[jira] [Commented] (CASSANDRA-13009) Speculative retry bugs

[Simon Zhou (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832727#comment-15832727
 ] 

Simon Zhou commented on CASSANDRA-13009:


[~tjake], do you mind reviewing this patch? Thanks.

> Speculative retry bugs
> --
>
> Key: CASSANDRA-13009
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13009
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
> Fix For: 3.0.11
>
> Attachments: CASSANDRA-13009-v1.patch
>
>
> There are a few issues with speculative retry:
> 1. Time unit bugs. These are from ColumnFamilyStore (v3.0.10):
> The left hand side is in nanos, as the name suggests, while the right hand 
> side is in millis.
> {code}
> sampleLatencyNanos = DatabaseDescriptor.getReadRpcTimeout() / 2;
> {code}
> Here coordinatorReadLatency is already in nanos and we shouldn't multiple the 
> value by 1000. This was a regression in 8896a70 when we switch metrics 
> library and the two libraries use different time units.
> {code}
> sampleLatencyNanos = (long) 
> (metric.coordinatorReadLatency.getSnapshot().getValue(retryPolicy.threshold())
>  * 1000d);
> {code}
> 2. Confusing overload protection and retry delay. As the name 
> "sampleLatencyNanos" suggests, it should be used to keep the actually sampled 
> read latency. However, we assign it the retry threshold in the case of 
> CUSTOM. Then we compare the retry threshold with read timeout (defaults to 
> 5000ms). This means, if we use speculative_retry=10ms for the table, we won't 
> be able to avoid being overloaded. We should compare the actual read latency 
> with the read timeout for overload protection. See line 450 of 
> ColumnFamilyStore.java and line 279 of AbstractReadExecutor.java.
> My proposals are:
> a. We use sampled p99 delay and compare it with a customizable threshold 
> (-Dcassandra.overload.threshold) for overload detection.
> b. Introduce another variable retryDelayNanos for waiting time before retry. 
> This is the value from table setting (PERCENTILE or CUSTOM).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13009) Speculative retry bugs

[Simon Zhou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Zhou updated CASSANDRA-13009:
---
Reviewer:   (was: T Jake Luciani)

> Speculative retry bugs
> --
>
> Key: CASSANDRA-13009
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13009
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
> Fix For: 3.0.11
>
> Attachments: CASSANDRA-13009-v1.patch
>
>
> There are a few issues with speculative retry:
> 1. Time unit bugs. These are from ColumnFamilyStore (v3.0.10):
> The left hand side is in nanos, as the name suggests, while the right hand 
> side is in millis.
> {code}
> sampleLatencyNanos = DatabaseDescriptor.getReadRpcTimeout() / 2;
> {code}
> Here coordinatorReadLatency is already in nanos and we shouldn't multiple the 
> value by 1000. This was a regression in 8896a70 when we switch metrics 
> library and the two libraries use different time units.
> {code}
> sampleLatencyNanos = (long) 
> (metric.coordinatorReadLatency.getSnapshot().getValue(retryPolicy.threshold())
>  * 1000d);
> {code}
> 2. Confusing overload protection and retry delay. As the name 
> "sampleLatencyNanos" suggests, it should be used to keep the actually sampled 
> read latency. However, we assign it the retry threshold in the case of 
> CUSTOM. Then we compare the retry threshold with read timeout (defaults to 
> 5000ms). This means, if we use speculative_retry=10ms for the table, we won't 
> be able to avoid being overloaded. We should compare the actual read latency 
> with the read timeout for overload protection. See line 450 of 
> ColumnFamilyStore.java and line 279 of AbstractReadExecutor.java.
> My proposals are:
> a. We use sampled p99 delay and compare it with a customizable threshold 
> (-Dcassandra.overload.threshold) for overload detection.
> b. Introduce another variable retryDelayNanos for waiting time before retry. 
> This is the value from table setting (PERCENTILE or CUSTOM).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: ninja-fix followup to CASSANDRA-4663

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4d67639d3 -> 85e4df4bb


ninja-fix followup to CASSANDRA-4663


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85e4df4b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85e4df4b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85e4df4b

Branch: refs/heads/trunk
Commit: 85e4df4bbf0ed5447c7328faa7495ad0b0f1cc22
Parents: 4d67639
Author: Jason Brown 
Authored: Fri Jan 20 13:46:56 2017 -0800
Committer: Jason Brown 
Committed: Fri Jan 20 13:46:56 2017 -0800

--
 test/unit/org/apache/cassandra/dht/BootStrapperTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/85e4df4b/test/unit/org/apache/cassandra/dht/BootStrapperTest.java
--
diff --git a/test/unit/org/apache/cassandra/dht/BootStrapperTest.java 
b/test/unit/org/apache/cassandra/dht/BootStrapperTest.java
index a2858dc..ba0352d 100644
--- a/test/unit/org/apache/cassandra/dht/BootStrapperTest.java
+++ b/test/unit/org/apache/cassandra/dht/BootStrapperTest.java
@@ -98,7 +98,7 @@ public class BootStrapperTest
 InetAddress myEndpoint = InetAddress.getByName("127.0.0.1");
 
 assertEquals(numOldNodes, tmd.sortedTokens().size());
-RangeStreamer s = new RangeStreamer(tmd, null, myEndpoint, 
"Bootstrap", true, DatabaseDescriptor.getEndpointSnitch(), new 
StreamStateStore(), false);
+RangeStreamer s = new RangeStreamer(tmd, null, myEndpoint, 
"Bootstrap", true, DatabaseDescriptor.getEndpointSnitch(), new 
StreamStateStore(), false, 1);
 IFailureDetector mockFailureDetector = new IFailureDetector()
 {
 public boolean isAlive(InetAddress ep)

[jira] [Comment Edited] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Soumya Sanyal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832423#comment-15832423
 ] 

Soumya Sanyal edited comment on CASSANDRA-11163 at 1/20/17 9:31 PM:


FWIW - this is causing startup on nodes in our cluster to take in excess of 
15mins. We do have roughly 1.5TB on disk. We're on a patched version of 3.9.

What I don't quite understand is why this behavior persists past the first 
restart of the BF FP?


was (Author: _soumya_):
FWIW - this is causing startup on nodes in our cluster to take in excess of 
15mins. We do have roughly 1.5TB on disk. We're on a patched version of 3.9

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
> Fix For: 3.x
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root 

[jira] [Comment Edited] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Soumya Sanyal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832423#comment-15832423
 ] 

Soumya Sanyal edited comment on CASSANDRA-11163 at 1/20/17 9:27 PM:


FWIW - this is causing startup on nodes in our cluster to take in excess of 
15mins. We do have roughly 1.5TB on disk. We're on a patched version of 3.9


was (Author: _soumya_):
FWIW - this is causing startup on nodes in our cluster to take in excess of 
15mins. We do have roughly 1.5TB on disk.

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
> Fix For: 3.x
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Soumya Sanyal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832423#comment-15832423
 ] 

Soumya Sanyal commented on CASSANDRA-11163:
---

FWIW - this is causing startup on nodes in our cluster to take in excess of 
15mins. We do have roughly 1.5TB on disk.

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
> Fix For: 3.x
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 root root 10808 Feb 12 00:03 ma-8-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 12 00:03 ma-8-big-Digest.crc32
> -rw-r--r-- 1 root root  19660275 Feb 12 00:03 ma-8-big-Data.db
> -rw-r--r-- 1 root root  1204 Feb 12 

[jira] [Updated] (CASSANDRA-12847) cqlsh DESCRIBE output doesn't properly quote index names

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-12847:

Fix Version/s: (was: 2.2.x)
   Status: Patch Available  (was: In Progress)

> cqlsh DESCRIBE output doesn't properly quote index names
> 
>
> Key: CASSANDRA-12847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12847
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>  Labels: cqlsh
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-8365 fixed the CQL grammar so that quoting index names preserves 
> case. The output of DESCRIBE in cqlsh wasn't updated however so this doesn't 
> round-trip properly. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12847) cqlsh DESCRIBE output doesn't properly quote index names

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832409#comment-15832409
 ] 

Sam Tunnicliffe commented on CASSANDRA-12847:
-

I've updated the bundled driver and kicked off CI runs (for 3.0 & 3.11 branches 
only as the trunk build is broken currently, I'll do the same for that when 
it's fixed). Also updated the dtest to only expect the new behaviour with the 
correct versions. 

||branch||testall||dtest||
|[12847-3.0|https://github.com/beobal/cassandra/tree/12847-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.0-dtest]|
|[12847-3.11|https://github.com/beobal/cassandra/tree/12847-3.11]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.11-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.11-dtest]|
|[12847-trunk|https://github.com/beobal/cassandra/tree/12847-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-trunk-dtest]|

cc [~aholmber] [~ifesdjeen] [~philipthompson]

> cqlsh DESCRIBE output doesn't properly quote index names
> 
>
> Key: CASSANDRA-12847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12847
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>  Labels: cqlsh
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-8365 fixed the CQL grammar so that quoting index names preserves 
> case. The output of DESCRIBE in cqlsh wasn't updated however so this doesn't 
> round-trip properly. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832400#comment-15832400
 ] 

Michael Shuler commented on CASSANDRA-13125:


Would this have something to do with it?
{noformat}
(cassandra-3.11)mshuler@hana:~/git/cassandra$ grep -r sigar-bin conf/
conf/cassandra-env.ps1:$env:JVM_OPTS = "$env:JVM_OPTS 
-Djava.library.path=""$env:CASSANDRA_HOME\lib\sigar-bin"""
conf/cassandra-env.sh:JVM_OPTS="$JVM_OPTS 
-Djava.library.path=$CASSANDRA_HOME/lib/sigar-bin"

(cassandra-3.11)mshuler@hana:~/git/cassandra$ grep -r sigar-bin test/conf/
(cassandra-3.11)mshuler@hana:~/git/cassandra$
{noformat}

> Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
> 
>
> Key: CASSANDRA-13125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13125
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhongxiang Zheng
>Assignee: Yasuharu Goto
>Priority: Critical
> Attachments: diff-a.patch, diff-b.patch
>
>
> I found that rows are splitting and duplicated after upgrading the cluster 
> from 2.1.x to 3.0.x.
> I found the way to reproduce the problem as below.
> {code}
> $ ccm create test -v 2.1.16 -n 3 -s   
> 
> Current cluster is now: test
> $ ccm node1 cqlsh  -e "CREATE KEYSPACE test WITH replication = 
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 
> set, value2 set);"
> # Upgrade node1
> $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # Insert a row through node1(3.0.10)
> $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"   
> # Insert a row through node2(2.1.16)
> $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" 
> # The row inserted from node1 is splitting
> $ ccm node1 cqlsh -e "SELECT * FROM test.test ;"
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> $ for i in 1 2; do ccm node${i} nodetool flush; done
> # Results of sstable2json of node2. The row inserted from node1(3.0.10) is 
> different from the row inserted from node2(2.1.16).
> $ ccm node2 json -k test -c test
> running
> ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db']
> -- test-test-ka-1-Data.db -
> [
> {"key": "aaa",
>  "cells": [["","",1484564624769577],
>["value1","value2:!",1484564624769576,"t",1484564624],
>["value1:616161","",1484564624769577],
>["value1:626262","",1484564624769577],
>["value2:636363","",1484564624769577],
>["value2:646464","",1484564624769577]]},
> {"key": "bbb",
>  "cells": [["","",1484564634508029],
>["value1:_","value1:!",1484564634508028,"t",1484564634],
>["value1:616161","",1484564634508029],
>["value1:626262","",1484564634508029],
>["value2:_","value2:!",1484564634508028,"t",1484564634],
>["value2:636363","",1484564634508029],
>["value2:646464","",1484564634508029]]}
> ]
> # Upgrade node2,3
> $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # After upgrade node2,3, the row inserted from node1 is splitting in node2,3
> $ ccm node2 cqlsh -e "SELECT * FROM test.test ;"  
>   
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> (3 rows)
> # Results of sstabledump
> # node1
> [
>   {
> "partition" : {
>   "key" : [ "aaa" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 17,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" },
> "cells" : [
>   { "name" : "value1", "deletion_info" : { "marked_deleted" : 
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } 
> },
>   { "name" : "value1", "path" : [ "aaa" ], "value" : "" },
>   { "name" : "value1", "path" : [ "bbb" ], "value" : "" },
>   { "name" : "value2", "deletion_info" : { "marked_deleted" : 
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } 
> },
>   { "name" : 

[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832384#comment-15832384
 ] 

Tyler Hobbs commented on CASSANDRA-13125:
-

[~slebresne] the patch and new test look good to me.  For some reason related 
to sigar your new unit test is erroring on Jenkins in the 3.11 version -- maybe 
[~mshuler] knows what's up with that?

> Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
> 
>
> Key: CASSANDRA-13125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13125
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhongxiang Zheng
>Assignee: Yasuharu Goto
>Priority: Critical
> Attachments: diff-a.patch, diff-b.patch
>
>
> I found that rows are splitting and duplicated after upgrading the cluster 
> from 2.1.x to 3.0.x.
> I found the way to reproduce the problem as below.
> {code}
> $ ccm create test -v 2.1.16 -n 3 -s   
> 
> Current cluster is now: test
> $ ccm node1 cqlsh  -e "CREATE KEYSPACE test WITH replication = 
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 
> set, value2 set);"
> # Upgrade node1
> $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # Insert a row through node1(3.0.10)
> $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"   
> # Insert a row through node2(2.1.16)
> $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" 
> # The row inserted from node1 is splitting
> $ ccm node1 cqlsh -e "SELECT * FROM test.test ;"
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> $ for i in 1 2; do ccm node${i} nodetool flush; done
> # Results of sstable2json of node2. The row inserted from node1(3.0.10) is 
> different from the row inserted from node2(2.1.16).
> $ ccm node2 json -k test -c test
> running
> ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db']
> -- test-test-ka-1-Data.db -
> [
> {"key": "aaa",
>  "cells": [["","",1484564624769577],
>["value1","value2:!",1484564624769576,"t",1484564624],
>["value1:616161","",1484564624769577],
>["value1:626262","",1484564624769577],
>["value2:636363","",1484564624769577],
>["value2:646464","",1484564624769577]]},
> {"key": "bbb",
>  "cells": [["","",1484564634508029],
>["value1:_","value1:!",1484564634508028,"t",1484564634],
>["value1:616161","",1484564634508029],
>["value1:626262","",1484564634508029],
>["value2:_","value2:!",1484564634508028,"t",1484564634],
>["value2:636363","",1484564634508029],
>["value2:646464","",1484564634508029]]}
> ]
> # Upgrade node2,3
> $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # After upgrade node2,3, the row inserted from node1 is splitting in node2,3
> $ ccm node2 cqlsh -e "SELECT * FROM test.test ;"  
>   
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> (3 rows)
> # Results of sstabledump
> # node1
> [
>   {
> "partition" : {
>   "key" : [ "aaa" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 17,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" },
> "cells" : [
>   { "name" : "value1", "deletion_info" : { "marked_deleted" : 
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } 
> },
>   { "name" : "value1", "path" : [ "aaa" ], "value" : "" },
>   { "name" : "value1", "path" : [ "bbb" ], "value" : "" },
>   { "name" : "value2", "deletion_info" : { "marked_deleted" : 
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } 
> },
>   { "name" : "value2", "path" : [ "ccc" ], "value" : "" },
>   { "name" : "value2", "path" : [ "ddd" ], "value" : "" }
> ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "bbb" ],
>   "position" : 48
> },
> "rows" : [
>   {
> 

[jira] [Commented] (CASSANDRA-13143) Cassandra can accept invalid durations

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832371#comment-15832371
 ] 

Benjamin Lerer commented on CASSANDRA-13143:


[~thobbs] Could you also review that ticket?

> Cassandra can accept invalid durations
> --
>
> Key: CASSANDRA-13143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13143
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>
> A duration can be positive or negative. If the duration is positive the 
> months, days and nanoseconds must be greater or equals to zero. If the 
> duration is negative the months, days and nanoseconds must be smaller or 
> equals to zero.
> Currently, it is possible to send to C* a duration which does not respect 
> that rule and the data will not be reject.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13143) Cassandra can accept invalid durations

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13143:
---
Assignee: Benjamin Lerer
  Status: Patch Available  (was: Open)

> Cassandra can accept invalid durations
> --
>
> Key: CASSANDRA-13143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13143
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>
> A duration can be positive or negative. If the duration is positive the 
> months, days and nanoseconds must be greater or equals to zero. If the 
> duration is negative the months, days and nanoseconds must be smaller or 
> equals to zero.
> Currently, it is possible to send to C* a duration which does not respect 
> that rule and the data will not be reject.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13143) Cassandra can accept invalid durations

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832370#comment-15832370
 ] 

Benjamin Lerer commented on CASSANDRA-13143:


||3.11|[branch|https://github.com/apache/cassandra/compare/trunk...blerer:13143-3.11]|[utest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13143-3.11-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13143-3.11-dtest/]|

The patch fix the validation and add some tests for it.

> Cassandra can accept invalid durations
> --
>
> Key: CASSANDRA-13143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13143
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>
> A duration can be positive or negative. If the duration is positive the 
> months, days and nanoseconds must be greater or equals to zero. If the 
> duration is negative the months, days and nanoseconds must be smaller or 
> equals to zero.
> Currently, it is possible to send to C* a duration which does not respect 
> that rule and the data will not be reject.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13143) Cassandra can accept invalid durations

[Benjamin Lerer (JIRA)]

Benjamin Lerer created CASSANDRA-13143:
--

 Summary: Cassandra can accept invalid durations
 Key: CASSANDRA-13143
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13143
 Project: Cassandra
  Issue Type: Bug
Reporter: Benjamin Lerer


A duration can be positive or negative. If the duration is positive the months, 
days and nanoseconds must be greater or equals to zero. If the duration is 
negative the months, days and nanoseconds must be smaller or equals to zero.
Currently, it is possible to send to C* a duration which does not respect that 
rule and the data will not be reject.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12850) Add the duration type to the protocol V5

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12850:

Reviewer: Tyler Hobbs

> Add the duration type to the protocol V5
> 
>
> Key: CASSANDRA-12850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12850
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The Duration type need to be added to the protocol specifications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12443) Remove alter type support

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer resolved CASSANDRA-12443.

Resolution: Fixed

Committed the followup patch into 3.0 at 
f3b452c54dea0366436ebe2bceac747bdf599e90 and merged into 3.11 and trunk

> Remove alter type support
> -
>
> Key: CASSANDRA-12443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12443
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Benjamin Lerer
> Fix For: 3.0.11, 3.10
>
>
> Currently, we allow altering of types. However, because we no longer store 
> the length for all types anymore, switching from a fixed-width to 
> variable-width type causes issues. commitlog playback breaking startup, 
> queries currently in flight getting back bad results, and special casing 
> required to handle the changes. In addition, this would solve 
> CASSANDRA-10309, as there is no possibility of the types changing while an 
> SSTableReader is open.
> For fixed-length, compatible types, the alter also doesn't add much over a 
> cast, so users could use that in order to retrieve the altered type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-4663) Parallelize streaming of different keyspaces for bootstrap and rebuild

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-4663:
---
   Resolution: Fixed
 Assignee: Corentin Chary
Fix Version/s: (was: 3.x)
   4.0
   Status: Resolved  (was: Patch Available)

> Parallelize streaming of different keyspaces for bootstrap and rebuild
> --
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Corentin Chary
>Priority: Minor
> Fix For: 4.0
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-4663) Parallelize streaming of different keyspaces for bootstrap and rebuild

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832322#comment-15832322
 ] 

Jason Brown commented on CASSANDRA-4663:


committed as sha {{4d67639d38b3e3a6fd0a3487a99b9755abda469d}} to 4.0. Thanks!

> Parallelize streaming of different keyspaces for bootstrap and rebuild
> --
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Parallelize streaming of different keyspaces for bootstrap and rebuild

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk d3704d8a0 -> 4d67639d3


Parallelize streaming of different keyspaces for bootstrap and rebuild

patch by Corentin Chary; reviewed by jasobrown for CASSANDRA-4663


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d67639d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d67639d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d67639d

Branch: refs/heads/trunk
Commit: 4d67639d38b3e3a6fd0a3487a99b9755abda469d
Parents: d3704d8
Author: Corentin Chary 
Authored: Wed Dec 7 11:11:06 2016 +0100
Committer: Jason Brown 
Committed: Fri Jan 20 11:47:18 2017 -0800

--
 CHANGES.txt  | 1 +
 conf/cassandra.yaml  | 6 ++
 src/java/org/apache/cassandra/config/Config.java | 1 +
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 5 +
 src/java/org/apache/cassandra/dht/BootStrapper.java  | 3 ++-
 src/java/org/apache/cassandra/dht/RangeStreamer.java | 7 +--
 src/java/org/apache/cassandra/service/StorageService.java| 3 ++-
 7 files changed, 22 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ebf161d..86ecbc4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Parallelize streaming of different keyspaces (4663)
  * Improved compactions metrics (CASSANDRA-13015)
  * Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
  * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 4891706..e728796 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -817,6 +817,12 @@ cross_node_timeout: false
 # times out in 10 minutes by default
 # streaming_keep_alive_period_in_secs: 300
 
+# Limit number of connections per host for streaming
+# Increase this when you notice that joins are CPU-bound rather that network
+# bound (for example a few nodes with big files).
+# streaming_connections_per_host: 1
+
+
 # phi value that must be reached for a host to be marked down.
 # most users should never need to adjust this.
 # phi_convict_threshold: 8

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index f5a8722..6fb999e 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -101,6 +101,7 @@ public class Config
 @Deprecated
 public int streaming_socket_timeout_in_ms = 8640; //24 hours
 
+public Integer streaming_connections_per_host = 1;
 public Integer streaming_keep_alive_period_in_secs = 300; //5 minutes
 
 public boolean cross_node_timeout = false;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index c43672a..5aa7065 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -2030,6 +2030,11 @@ public class DatabaseDescriptor
 return conf.streaming_keep_alive_period_in_secs;
 }
 
+public static int getStreamingConnectionsPerHost()
+{
+return conf.streaming_connections_per_host;
+}
+
 public static String getLocalDataCenter()
 {
 return localDC;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/src/java/org/apache/cassandra/dht/BootStrapper.java
--
diff --git a/src/java/org/apache/cassandra/dht/BootStrapper.java 
b/src/java/org/apache/cassandra/dht/BootStrapper.java
index 1e00f48..15e75fe 100644
--- a/src/java/org/apache/cassandra/dht/BootStrapper.java
+++ b/src/java/org/apache/cassandra/dht/BootStrapper.java
@@ -77,7 +77,8 @@ public class BootStrapper extends ProgressEventNotifierSupport
useStrictConsistency,

[jira] [Updated] (CASSANDRA-4663) Parallelize streaming of different keyspaces for bootstrap and rebuild

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-4663:
---
Summary: Parallelize streaming of different keyspaces for bootstrap and 
rebuild  (was: Parallelize streaming of different keyspaces)

> Parallelize streaming of different keyspaces for bootstrap and rebuild
> --
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-4663) Parallelize streaming of different keyspaces

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-4663:
---
Summary: Parallelize streaming of different keyspaces  (was: Streaming 
sends one file at a time serially. )

> Parallelize streaming of different keyspaces
> 
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832314#comment-15832314
 ] 

Anubhav Kale commented on CASSANDRA-4663:
-

OOF 1/20










> Streaming sends one file at a time serially. 
> -
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832312#comment-15832312
 ] 

Jason Brown commented on CASSANDRA-4663:


OK, test look reasonable (no new dtest failures that are not currently failing 
on trunk). For the record, [~iksaif]'s patch parallelizes the streaming of the 
keyspaces being sent. If you have multiple keyspaces, you will potentially get 
an nice win. However, if there is only one keyspace (or most data is in one), 
there is no parallelization gained. I'll update the title of this jira to 
reflect that. As part of CASSANDRA-12229, or an immediate follow up, I'll be 
addressing sending sending files in parallel, so that request won't fall off 
the radar.

+1'ing the ticket and will commit briefly

> Streaming sends one file at a time serially. 
> -
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Sotirios Delimanolis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832261#comment-15832261
 ] 

Sotirios Delimanolis edited comment on CASSANDRA-12100 at 1/20/17 7:13 PM:
---

Any chance you can backport this to 2.2.9? (Referring to [this 
commit|https://github.com/apache/cassandra/commit/05483a962c64c350315fc738c697980b22361cc3].)


was (Author: s_delima):
Any chance you can backport this to 2.2.9?

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 3.0.9, 3.8
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Sotirios Delimanolis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832261#comment-15832261
 ] 

Sotirios Delimanolis commented on CASSANDRA-12100:
--

Any chance you can backport this to 2.2.9?

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 3.0.9, 3.8
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9

[Yasuharu Goto (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832169#comment-15832169
 ] 

Yasuharu Goto commented on CASSANDRA-13125:
---

Thank you [~slebresne]! I checked that your patches worked properly in my 
reproduce procedure. The result is below. Now I could see that C*3.0 generated 
{{[c-c!][d-d!]}} style range tombstones and the rows are not broken!

h4. On 13125-3.0
{noformat}
cqlsh> insert into test.test(a,b,c,d,e) values(14,1,{2,3},{4,5},6);
cqlsh> select * from test.test;

 a  | b | c  | d  | e
+---+++---
 14 | 1 | {2, 3} | {4, 5} | 6

RangeTombstone(0), 
start:org.apache.cassandra.db.composites.CompoundSparseCellName@78e3b54a, 
end:org.apache.cassandra.db.composites.BoundedComposite@6b517b57, 
markedAt:1484931972134334, delTime:1484931972
RangeTombstone(1), 
start:org.apache.cassandra.db.composites.CompoundSparseCellName@78e3b54b, 
end:org.apache.cassandra.db.composites.BoundedComposite@6b517b58, 
markedAt:1484931972134334, delTime:1484931972
DeletionInfo:{deletedAt=-9223372036854775808, localDeletion=2147483647, 
ranges=[c-c:!, deletedAt=1484931972134334, localDeletion=1484931972][d-d:!, 
deletedAt=1484931972134334, localDeletion=1484931972]}
from:/127.0.0.1, payload:Mutation(keyspace='test', key='000e', 
modifications=[ColumnFamily(test -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[c-c:!, deletedAt=1484931972134334, 
localDeletion=1484931972][d-d:!, deletedAt=1484931972134334, 
localDeletion=1484931972]}- 
[:false:0@1484931972134335,b:false:4@1484931972134335,c:0002:false:0@1484931972134335,c:0003:false:0@1484931972134335,d:0004:false:0@1484931972134335,d:0005:false:0@1484931972134335,e:false:4@1484931972134335,])]),
 verb:MUTATION, version:8
{noformat}

h4. On 13125-3.11
{noformat}
cqlsh> insert into test.test(a,b,c,d,e) values(14,1,{2,3},{4,5},6);
cqlsh> select * from test.test;   
 a  | b | c  | d  | e
+---+++---
 14 | 1 | {2, 3} | {4, 5} | 6

Mutation.deserialize() size==1
RangeTombstone(0), 
start:org.apache.cassandra.db.composites.CompoundSparseCellName@4316af5d, 
end:org.apache.cassandra.db.composites.BoundedComposite@256e93b0, 
markedAt:1484933162431359, delTime:1484933162
RangeTombstone(1), 
start:org.apache.cassandra.db.composites.CompoundSparseCellName@4316af5e, 
end:org.apache.cassandra.db.composites.BoundedComposite@256e93b1, 
markedAt:1484933162431359, delTime:1484933162
DeletionInfo:{deletedAt=-9223372036854775808, localDeletion=2147483647, 
ranges=[c-c:!, deletedAt=1484933162431359, localDeletion=1484933162][d-d:!, 
deletedAt=1484933162431359, localDeletion=1484933162]}
from:/127.0.0.1, payload:Mutation(keyspace='test', key='000e', 
modifications=[ColumnFamily(test -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[c-c:!, deletedAt=1484933162431359, 
localDeletion=1484933162][d-d:!, deletedAt=1484933162431359, 
localDeletion=1484933162]}- 
[:false:0@1484933162431360,b:false:4@1484933162431360,c:0002:false:0@1484933162431360,c:0003:false:0@1484933162431360,d:0004:false:0@1484933162431360,d:0005:false:0@1484933162431360,e:false:4@1484933162431360,])]),
 verb:MUTATION, version:8
{noformat}



> Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
> 
>
> Key: CASSANDRA-13125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13125
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhongxiang Zheng
>Assignee: Yasuharu Goto
>Priority: Critical
> Attachments: diff-a.patch, diff-b.patch
>
>
> I found that rows are splitting and duplicated after upgrading the cluster 
> from 2.1.x to 3.0.x.
> I found the way to reproduce the problem as below.
> {code}
> $ ccm create test -v 2.1.16 -n 3 -s   
> 
> Current cluster is now: test
> $ ccm node1 cqlsh  -e "CREATE KEYSPACE test WITH replication = 
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 
> set, value2 set);"
> # Upgrade node1
> $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # Insert a row through node1(3.0.10)
> $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"   
> # Insert a row through node2(2.1.16)
> $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" 
> # The row inserted from node1 is splitting
> $ ccm node1 cqlsh -e "SELECT * FROM test.test ;"
>  id  | value1 | value2
> 

[jira] [Comment Edited] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

[Tom van der Woerdt (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832087#comment-15832087
 ] 

Tom van der Woerdt edited comment on CASSANDRA-13142 at 1/20/17 4:49 PM:
-

Cancelling view/index builds is not a "minor" issue. :) Granted, a restart 
should resume the view build, but that's not obvious to operators at all.


was (Author: tvdw):
Cancelling view/index builds is not a "minor" issue. :)

> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.
> I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
> for debate. The main negative effect is annoying operators (which I tend to 
> rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

[Tom van der Woerdt (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832087#comment-15832087
 ] 

Tom van der Woerdt commented on CASSANDRA-13142:


Cancelling view/index builds is not a "minor" issue. :)

> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.
> I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
> for debate. The main negative effect is annoying operators (which I tend to 
> rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832084#comment-15832084
 ] 

Kurt Greaves commented on CASSANDRA-13142:
--

I've already done all the necessary reconnaissance so will write a patch soon, 
I think best solution would be to provide a more modular way to specify 
compaction types to "ignore" in 
{{CompactionManager.interruptCompactionsForCFs()}}. 

> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.
> I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
> for debate. The main negative effect is annoying operators (which I tend to 
> rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

[Kurt Greaves (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves reassigned CASSANDRA-13142:


Assignee: Kurt Greaves

> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.
> I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
> for debate. The main negative effect is annoying operators (which I tend to 
> rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12243) upgradesstables cancels view build

[Kurt Greaves (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves resolved CASSANDRA-12243.
--
Resolution: Duplicate

Marking as duplicate in favor of CASSANDRA-13142 as it encapsulates this.

> upgradesstables cancels view build
> --
>
> Key: CASSANDRA-12243
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12243
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>
> It appears that running upgradesstables cancels view builds, even if there 
> are no sstables to upgrade :
> {code}
> $ nodetool compactionstats -H
> pending tasks: 54
>  id   compaction type   keyspace 
> table   completedtotal unit   progress
>1cfee290-4dbe-11e6-b619-fb674e694d11View build  mykeyspace   
> mytable   666 bytes   1015 bytes   ranges 65.62%
> Active compaction remaining time :n/a
> $ nodetool upgradesstables
> $ nodetool compactionstats -H
> pending tasks: 53
> {code}
> Easy to reproduce.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

[Kurt Greaves (JIRA)]

Kurt Greaves created CASSANDRA-13142:


 Summary: Upgradesstables cancels compactions unnecessarily
 Key: CASSANDRA-13142
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
 Project: Cassandra
  Issue Type: Bug
Reporter: Kurt Greaves
Priority: Minor


Since at least 1.2 upgradesstables will cancel any compactions bar validations 
when run. This was originally determined as a non-issue in CASSANDRA-3430 
however can be quite annoying (especially with STCS) as a compaction will 
output the new version anyway. Furthermore, as per CASSANDRA-12243 it also 
stops things like view builds and I assume secondary index builds as well which 
is not ideal.

We should avoid cancelling compactions unnecessarily.

I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
for debate. The main negative effect is annoying operators (which I tend to 
rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13141) test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test

[Sean McCarthy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean McCarthy updated CASSANDRA-13141:
--
Component/s: Testing

> test failure in 
> cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test
> -
>
> Key: CASSANDRA-13141
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13141
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cql_tests/MiscellaneousCQLTester/prepared_statement_invalidation_test
> {code}
> Error Message
> Error from server: code=2200 [Invalid query] message="Altering of types is 
> not allowed"
> {code}{code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tests.py", line 633, in 
> prepared_statement_invalidation_test
> session.execute("ALTER TABLE test ALTER d TYPE blob")
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state).result()
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 3784, in result
> raise self._final_exception
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13141) test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-13141:
-

 Summary: test failure in 
cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test
 Key: CASSANDRA-13141
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13141
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean McCarthy
 Attachments: node1_debug.log, node1_gc.log, node1.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cql_tests/MiscellaneousCQLTester/prepared_statement_invalidation_test

{code}
Error Message

Error from server: code=2200 [Invalid query] message="Altering of types is not 
allowed"
{code}{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/cql_tests.py", line 633, in 
prepared_statement_invalidation_test
session.execute("ALTER TABLE test ALTER d TYPE blob")
  File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 1998, 
in execute
return self.execute_async(query, parameters, trace, custom_payload, 
timeout, execution_profile, paging_state).result()
  File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 3784, 
in result
raise self._final_exception
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13140) test failure in cqlsh_tests.cqlsh_tests.CqlshSmokeTest.test_alter_table

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-13140:
-

 Summary: test failure in 
cqlsh_tests.cqlsh_tests.CqlshSmokeTest.test_alter_table
 Key: CASSANDRA-13140
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13140
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean McCarthy
 Attachments: node1_debug.log, node1_gc.log, node1.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cqlsh_tests.cqlsh_tests/CqlshSmokeTest/test_alter_table

{code}
Error Message

[u'test', u'i', u'ascii'] unexpectedly found in [[u'test', u'key', u'text'], 
[u'test', u'i', u'ascii']]
{code}{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 1839, 
in test_alter_table
self.assertNotIn(old_column_spec, new_columns)
  File "/usr/lib/python2.7/unittest/case.py", line 810, in assertNotIn
self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13139) test failure in hintedhandoff_test.TestHintedHandoff.hintedhandoff_decom_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-13139:
-

 Summary: test failure in 
hintedhandoff_test.TestHintedHandoff.hintedhandoff_decom_test
 Key: CASSANDRA-13139
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13139
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Sean McCarthy
 Attachments: node1_debug.log, node1_gc.log, node1.log, 
node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
node3.log, node4_debug.log, node4_gc.log, node4.log

example failure:

http://cassci.datastax.com/job/trunk_novnode_dtest/503/testReport/hintedhandoff_test/TestHintedHandoff/hintedhandoff_decom_test

{code}
Error Message

Subprocess ['nodetool', '-h', 'localhost', '-p', '7200', ['decommission']] 
exited with non-zero status; exit status: 1; 
stdout: nodetool: Unsupported operation: Not enough live nodes to maintain 
replication factor in keyspace system_distributed (RF = 3, N = 3). Perform a 
forceful decommission to ignore.
See 'nodetool help' or 'nodetool help '.
{code}{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/hintedhandoff_test.py", line 169, in 
hintedhandoff_decom_test
node2.decommission()
  File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 1314, in 
decommission
self.nodetool("decommission")
  File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 783, in 
nodetool
return handle_external_tool_process(p, ['nodetool', '-h', 'localhost', 
'-p', str(self.jmx_port), cmd.split()])
  File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 1993, in 
handle_external_tool_process
raise ToolError(cmd_args, rc, out, err)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Follow up 12443

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 e3d26b6aa -> 1a56dd063


Follow up 12443


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3b452c5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3b452c5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3b452c5

Branch: refs/heads/cassandra-3.11
Commit: f3b452c54dea0366436ebe2bceac747bdf599e90
Parents: 7a06df7
Author: Benjamin Lerer 
Authored: Fri Jan 20 15:53:34 2017 +0100
Committer: Benjamin Lerer 
Committed: Fri Jan 20 15:53:34 2017 +0100

--
 src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3b452c5/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index a3370dc..44f3a96 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -84,9 +84,9 @@ public final class CFMetaData
 private final boolean isView;
 private final boolean isIndex;
 
-public final ClusteringComparator comparator;  // bytes, long, timeuuid, 
utf8, etc. This is built directly from clusteringColumns
+public volatile ClusteringComparator comparator;  // bytes, long, 
timeuuid, utf8, etc. This is built directly from clusteringColumns
 public final IPartitioner partitioner;// partitioner the table 
uses
-private final AbstractType keyValidator;
+private volatile AbstractType keyValidator;
 
 private final Serializers serializers;
 
@@ -285,10 +285,6 @@ public final class CFMetaData
 
 this.serializers = new Serializers(this);
 
-this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
-List keyTypes = extractTypes(partitionKeyColumns);
-this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
-
 rebuild();
 }
 
@@ -296,6 +292,8 @@ public final class CFMetaData
 // are kept because they are often useful in a different format.
 private void rebuild()
 {
+this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
+
 Map newColumnMetadata = new HashMap<>();
 for (ColumnDefinition def : partitionKeyColumns)
 newColumnMetadata.put(def.name.bytes, def);
@@ -306,6 +304,9 @@ public final class CFMetaData
 
 this.columnMetadata = newColumnMetadata;
 
+List keyTypes = extractTypes(partitionKeyColumns);
+this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
+
 if (isCompactTable())
 this.compactValueColumn = 
CompactTables.getCompactValueColumn(partitionColumns, isSuper());
 }

[3/3] cassandra git commit: Merge branch cassandra-3.11 into trunk

[blerer]

Merge branch cassandra-3.11 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3704d8a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3704d8a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3704d8a

Branch: refs/heads/trunk
Commit: d3704d8a06927e234250463796fc515e2b14d2cb
Parents: d6da7b7 1a56dd0
Author: Benjamin Lerer 
Authored: Fri Jan 20 16:03:30 2017 +0100
Committer: Benjamin Lerer 
Committed: Fri Jan 20 16:03:37 2017 +0100

--
 src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3704d8a/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index 097f29d,d0932ed..c8180f3
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -89,10 -90,12 +89,10 @@@ public final class CFMetaDat
  private final boolean isView;
  private final boolean isIndex;
  
- public final ClusteringComparator comparator;  // bytes, long, timeuuid, 
utf8, etc. This is built directly from clusteringColumns
+ public volatile ClusteringComparator comparator;  // bytes, long, 
timeuuid, utf8, etc. This is built directly from clusteringColumns
  public final IPartitioner partitioner;// partitioner the 
table uses
- private final AbstractType keyValidator;
+ private volatile AbstractType keyValidator;
  
 -private final Serializers serializers;
 -
  // non-final, for now
  public volatile TableParams params = TableParams.DEFAULT;
  
@@@ -315,8 -320,13 +313,11 @@@
  
  this.columnMetadata = newColumnMetadata;
  
+ List keyTypes = extractTypes(partitionKeyColumns);
+ this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
+ 
  if (isCompactTable())
  this.compactValueColumn = 
CompactTables.getCompactValueColumn(partitionColumns, isSuper());
 -
 -this.allColumnFilter = ColumnFilter.all(this);
  }
  
  public Indexes getIndexes()

[1/3] cassandra git commit: Follow up 12443

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk d6da7b799 -> d3704d8a0


Follow up 12443


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3b452c5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3b452c5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3b452c5

Branch: refs/heads/trunk
Commit: f3b452c54dea0366436ebe2bceac747bdf599e90
Parents: 7a06df7
Author: Benjamin Lerer 
Authored: Fri Jan 20 15:53:34 2017 +0100
Committer: Benjamin Lerer 
Committed: Fri Jan 20 15:53:34 2017 +0100

--
 src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3b452c5/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index a3370dc..44f3a96 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -84,9 +84,9 @@ public final class CFMetaData
 private final boolean isView;
 private final boolean isIndex;
 
-public final ClusteringComparator comparator;  // bytes, long, timeuuid, 
utf8, etc. This is built directly from clusteringColumns
+public volatile ClusteringComparator comparator;  // bytes, long, 
timeuuid, utf8, etc. This is built directly from clusteringColumns
 public final IPartitioner partitioner;// partitioner the table 
uses
-private final AbstractType keyValidator;
+private volatile AbstractType keyValidator;
 
 private final Serializers serializers;
 
@@ -285,10 +285,6 @@ public final class CFMetaData
 
 this.serializers = new Serializers(this);
 
-this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
-List keyTypes = extractTypes(partitionKeyColumns);
-this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
-
 rebuild();
 }
 
@@ -296,6 +292,8 @@ public final class CFMetaData
 // are kept because they are often useful in a different format.
 private void rebuild()
 {
+this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
+
 Map newColumnMetadata = new HashMap<>();
 for (ColumnDefinition def : partitionKeyColumns)
 newColumnMetadata.put(def.name.bytes, def);
@@ -306,6 +304,9 @@ public final class CFMetaData
 
 this.columnMetadata = newColumnMetadata;
 
+List keyTypes = extractTypes(partitionKeyColumns);
+this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
+
 if (isCompactTable())
 this.compactValueColumn = 
CompactTables.getCompactValueColumn(partitionColumns, isSuper());
 }

[jira] [Commented] (CASSANDRA-13123) Draining a node might fail to delete all inactive commitlogs

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831886#comment-15831886
 ] 

Jason Brown commented on CASSANDRA-13123:
-

Pushed code for cassci to run tests. I think the change to 
{{CommitLogSegmentManager}} is probably legit, but I want to look at the test a 
little more before +1'ing it

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13123-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13123-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13123-trunk]|
|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.11-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-trunk-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.11-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-trunk-testall/]|


> Draining a node might fail to delete all inactive commitlogs
> 
>
> Key: CASSANDRA-13123
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13123
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Jan Urbański
>Assignee: Jan Urbański
> Fix For: 3.8
>
> Attachments: 13123-2.2.8.txt, 13123-3.0.10.txt, 13123-3.9.txt, 
> 13123-trunk.txt
>
>
> After issuing a drain command, it's possible that not all of the inactive 
> commitlogs are removed.
> The drain command shuts down the CommitLog instance, which in turn shuts down 
> the CommitLogSegmentManager. This has the effect of discarding any pending 
> management tasks it might have, like the removal of inactive commitlogs.
> This in turn leads to an excessive amount of commitlogs being left behind 
> after a drain and a lengthy recovery after a restart. With a fleet of dozens 
> of nodes, each of them leaving several GB of commitlogs after a drain and 
> taking up to two minutes to recover them on restart, the additional time 
> required to restart the entire fleet becomes noticeable.
> This problem is not present in 3.x or trunk because of the CLSM rewrite done 
> in CASSANDRA-8844.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a56dd06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a56dd06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a56dd06

Branch: refs/heads/trunk
Commit: 1a56dd063d8d032c33427013ff0e990f4e5b4760
Parents: e3d26b6 f3b452c
Author: Benjamin Lerer 
Authored: Fri Jan 20 16:02:27 2017 +0100
Committer: Benjamin Lerer 
Committed: Fri Jan 20 16:02:27 2017 +0100

--
 src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a56dd06/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index 1d3bb3a,44f3a96..d0932ed
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -297,22 -283,18 +297,20 @@@ public final class CFMetaDat
  this.clusteringColumns = clusteringColumns;
  this.partitionColumns = partitionColumns;
  
- this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
- List keyTypes = extractTypes(partitionKeyColumns);
- this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
 -this.serializers = new Serializers(this);
--
  rebuild();
 +
 +this.resource = DataResource.table(ksName, cfName);
 +this.serializers = new Serializers(this);
  }
  
  // This rebuild informations that are intrinsically duplicate of the 
table definition but
  // are kept because they are often useful in a different format.
  private void rebuild()
  {
+ this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
+ 
 -Map newColumnMetadata = new HashMap<>();
 +Map newColumnMetadata = 
Maps.newHashMapWithExpectedSize(partitionKeyColumns.size() + 
clusteringColumns.size() + partitionColumns.size());
 +
  for (ColumnDefinition def : partitionKeyColumns)
  newColumnMetadata.put(def.name.bytes, def);
  for (ColumnDefinition def : clusteringColumns)
@@@ -322,10 -304,11 +320,13 @@@
  
  this.columnMetadata = newColumnMetadata;
  
+ List keyTypes = extractTypes(partitionKeyColumns);
+ this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
+ 
  if (isCompactTable())
  this.compactValueColumn = 
CompactTables.getCompactValueColumn(partitionColumns, isSuper());
 +
 +this.allColumnFilter = ColumnFilter.all(this);
  }
  
  public Indexes getIndexes()

[2/2] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a56dd06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a56dd06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a56dd06

Branch: refs/heads/cassandra-3.11
Commit: 1a56dd063d8d032c33427013ff0e990f4e5b4760
Parents: e3d26b6 f3b452c
Author: Benjamin Lerer 
Authored: Fri Jan 20 16:02:27 2017 +0100
Committer: Benjamin Lerer 
Committed: Fri Jan 20 16:02:27 2017 +0100

--
 src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a56dd06/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --cc src/java/org/apache/cassandra/config/CFMetaData.java
index 1d3bb3a,44f3a96..d0932ed
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@@ -297,22 -283,18 +297,20 @@@ public final class CFMetaDat
  this.clusteringColumns = clusteringColumns;
  this.partitionColumns = partitionColumns;
  
- this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
- List keyTypes = extractTypes(partitionKeyColumns);
- this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
 -this.serializers = new Serializers(this);
--
  rebuild();
 +
 +this.resource = DataResource.table(ksName, cfName);
 +this.serializers = new Serializers(this);
  }
  
  // This rebuild informations that are intrinsically duplicate of the 
table definition but
  // are kept because they are often useful in a different format.
  private void rebuild()
  {
+ this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
+ 
 -Map newColumnMetadata = new HashMap<>();
 +Map newColumnMetadata = 
Maps.newHashMapWithExpectedSize(partitionKeyColumns.size() + 
clusteringColumns.size() + partitionColumns.size());
 +
  for (ColumnDefinition def : partitionKeyColumns)
  newColumnMetadata.put(def.name.bytes, def);
  for (ColumnDefinition def : clusteringColumns)
@@@ -322,10 -304,11 +320,13 @@@
  
  this.columnMetadata = newColumnMetadata;
  
+ List keyTypes = extractTypes(partitionKeyColumns);
+ this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
+ 
  if (isCompactTable())
  this.compactValueColumn = 
CompactTables.getCompactValueColumn(partitionColumns, isSuper());
 +
 +this.allColumnFilter = ColumnFilter.all(this);
  }
  
  public Indexes getIndexes()

cassandra git commit: Follow up 12443

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 7a06df79d -> f3b452c54


Follow up 12443


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3b452c5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3b452c5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3b452c5

Branch: refs/heads/cassandra-3.0
Commit: f3b452c54dea0366436ebe2bceac747bdf599e90
Parents: 7a06df7
Author: Benjamin Lerer 
Authored: Fri Jan 20 15:53:34 2017 +0100
Committer: Benjamin Lerer 
Committed: Fri Jan 20 15:53:34 2017 +0100

--
 src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3b452c5/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index a3370dc..44f3a96 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -84,9 +84,9 @@ public final class CFMetaData
 private final boolean isView;
 private final boolean isIndex;
 
-public final ClusteringComparator comparator;  // bytes, long, timeuuid, 
utf8, etc. This is built directly from clusteringColumns
+public volatile ClusteringComparator comparator;  // bytes, long, 
timeuuid, utf8, etc. This is built directly from clusteringColumns
 public final IPartitioner partitioner;// partitioner the table 
uses
-private final AbstractType keyValidator;
+private volatile AbstractType keyValidator;
 
 private final Serializers serializers;
 
@@ -285,10 +285,6 @@ public final class CFMetaData
 
 this.serializers = new Serializers(this);
 
-this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
-List keyTypes = extractTypes(partitionKeyColumns);
-this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
-
 rebuild();
 }
 
@@ -296,6 +292,8 @@ public final class CFMetaData
 // are kept because they are often useful in a different format.
 private void rebuild()
 {
+this.comparator = new 
ClusteringComparator(extractTypes(clusteringColumns));
+
 Map newColumnMetadata = new HashMap<>();
 for (ColumnDefinition def : partitionKeyColumns)
 newColumnMetadata.put(def.name.bytes, def);
@@ -306,6 +304,9 @@ public final class CFMetaData
 
 this.columnMetadata = newColumnMetadata;
 
+List keyTypes = extractTypes(partitionKeyColumns);
+this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : 
CompositeType.getInstance(keyTypes);
+
 if (isCompactTable())
 this.compactValueColumn = 
CompactTables.getCompactValueColumn(partitionColumns, isSuper());
 }

[jira] [Commented] (CASSANDRA-12443) Remove alter type support

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831854#comment-15831854
 ] 

Aleksey Yeschenko commented on CASSANDRA-12443:
---

Had a look offline already. +1

> Remove alter type support
> -
>
> Key: CASSANDRA-12443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12443
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Benjamin Lerer
> Fix For: 3.0.11, 3.10
>
>
> Currently, we allow altering of types. However, because we no longer store 
> the length for all types anymore, switching from a fixed-width to 
> variable-width type causes issues. commitlog playback breaking startup, 
> queries currently in flight getting back bad results, and special casing 
> required to handle the changes. In addition, this would solve 
> CASSANDRA-10309, as there is no possibility of the types changing while an 
> SSTableReader is open.
> For fixed-length, compatible types, the alter also doesn't add much over a 
> cast, so users could use that in order to retrieve the altered type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12443) Remove alter type support

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831853#comment-15831853
 ] 

Benjamin Lerer commented on CASSANDRA-12443:


The followup patch is 
[here|https://github.com/apache/cassandra/compare/trunk...blerer:12443-3.0-followup].

> Remove alter type support
> -
>
> Key: CASSANDRA-12443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12443
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Benjamin Lerer
> Fix For: 3.0.11, 3.10
>
>
> Currently, we allow altering of types. However, because we no longer store 
> the length for all types anymore, switching from a fixed-width to 
> variable-width type causes issues. commitlog playback breaking startup, 
> queries currently in flight getting back bad results, and special casing 
> required to handle the changes. In addition, this would solve 
> CASSANDRA-10309, as there is no possibility of the types changing while an 
> SSTableReader is open.
> For fixed-length, compatible types, the alter also doesn't add much over a 
> cast, so users could use that in order to retrieve the altered type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831816#comment-15831816
 ] 

Sylvain Lebresne commented on CASSANDRA-13125:
--

Thanks a lot for the very detailed reproduction case and analysis. You are 
absolutely right that it's abnormal for 3.0 to transform a range like 
{{\[c-c!\]\[d-d!\]}} into {{\[c-c!\]\[c-d!\]}}. And having duplicate row 
entries is only one of the problem this triggers, as with more collections you 
could this ending up to delete data that it shouldn't. In any case, the problem 
is definitively in 3.0 generating this in the first place, so that's what we 
should fix, so fixing this in LegacyRangeTombstone constructor (Plan-A above) 
is too late.

So the problem is when building the {{LegacyRangeTombstonList}} on 3.0, though 
it's not in the {{insertFrom}} method and the Plan-B patch would likely create 
other problems. The problem is in the {{LegacyBoundComparator}}: it delegates 
to {{clusteringComparator.compare()}} before it even checks if the bounds it 
compares have a {{collectionName}}, but that's incorrect.

Anyway, attaching below a fix for this as well as a simple unit test showing
where the problem lies:
| [13125-3.0|https://github.com/pcmanus/cassandra/commits/13125-3.0] | 
[utests|http://cassci.datastax.com/job/pcmanus-13125-3.0-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-13125-3.0-dtest] |
| [13125-3.11|https://github.com/pcmanus/cassandra/commits/13125-3.11] | 
[utests|http://cassci.datastax.com/job/pcmanus-13125-3.11-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-13125-3.11-dtest] |

[~Yasuharu], if you could check if this patch does properly work in your case, 
that would be much appreciated. [~thobbs], since you're familiar with 3.0 
compatibility code, would you also mind having a quick look at the patch?

> Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
> 
>
> Key: CASSANDRA-13125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13125
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhongxiang Zheng
>Assignee: Yasuharu Goto
>Priority: Critical
> Attachments: diff-a.patch, diff-b.patch
>
>
> I found that rows are splitting and duplicated after upgrading the cluster 
> from 2.1.x to 3.0.x.
> I found the way to reproduce the problem as below.
> {code}
> $ ccm create test -v 2.1.16 -n 3 -s   
> 
> Current cluster is now: test
> $ ccm node1 cqlsh  -e "CREATE KEYSPACE test WITH replication = 
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 
> set, value2 set);"
> # Upgrade node1
> $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # Insert a row through node1(3.0.10)
> $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"   
> # Insert a row through node2(2.1.16)
> $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" 
> # The row inserted from node1 is splitting
> $ ccm node1 cqlsh -e "SELECT * FROM test.test ;"
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> $ for i in 1 2; do ccm node${i} nodetool flush; done
> # Results of sstable2json of node2. The row inserted from node1(3.0.10) is 
> different from the row inserted from node2(2.1.16).
> $ ccm node2 json -k test -c test
> running
> ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db']
> -- test-test-ka-1-Data.db -
> [
> {"key": "aaa",
>  "cells": [["","",1484564624769577],
>["value1","value2:!",1484564624769576,"t",1484564624],
>["value1:616161","",1484564624769577],
>["value1:626262","",1484564624769577],
>["value2:636363","",1484564624769577],
>["value2:646464","",1484564624769577]]},
> {"key": "bbb",
>  "cells": [["","",1484564634508029],
>["value1:_","value1:!",1484564634508028,"t",1484564634],
>["value1:616161","",1484564634508029],
>["value1:626262","",1484564634508029],
>["value2:_","value2:!",1484564634508028,"t",1484564634],
>["value2:636363","",1484564634508029],
>["value2:646464","",1484564634508029]]}
> ]
> # Upgrade node2,3
> $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> 

[jira] [Updated] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13125:
-
Priority: Critical  (was: Major)

> Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
> 
>
> Key: CASSANDRA-13125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13125
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhongxiang Zheng
>Assignee: Yasuharu Goto
>Priority: Critical
> Attachments: diff-a.patch, diff-b.patch
>
>
> I found that rows are splitting and duplicated after upgrading the cluster 
> from 2.1.x to 3.0.x.
> I found the way to reproduce the problem as below.
> {code}
> $ ccm create test -v 2.1.16 -n 3 -s   
> 
> Current cluster is now: test
> $ ccm node1 cqlsh  -e "CREATE KEYSPACE test WITH replication = 
> {'class':'SimpleStrategy', 'replication_factor':3}"
> $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 
> set, value2 set);"
> # Upgrade node1
> $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # Insert a row through node1(3.0.10)
> $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});"   
> # Insert a row through node2(2.1.16)
> $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values 
> ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" 
> # The row inserted from node1 is splitting
> $ ccm node1 cqlsh -e "SELECT * FROM test.test ;"
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> $ for i in 1 2; do ccm node${i} nodetool flush; done
> # Results of sstable2json of node2. The row inserted from node1(3.0.10) is 
> different from the row inserted from node2(2.1.16).
> $ ccm node2 json -k test -c test
> running
> ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db']
> -- test-test-ka-1-Data.db -
> [
> {"key": "aaa",
>  "cells": [["","",1484564624769577],
>["value1","value2:!",1484564624769576,"t",1484564624],
>["value1:616161","",1484564624769577],
>["value1:626262","",1484564624769577],
>["value2:636363","",1484564624769577],
>["value2:646464","",1484564624769577]]},
> {"key": "bbb",
>  "cells": [["","",1484564634508029],
>["value1:_","value1:!",1484564634508028,"t",1484564634],
>["value1:616161","",1484564634508029],
>["value1:626262","",1484564634508029],
>["value2:_","value2:!",1484564634508028,"t",1484564634],
>["value2:636363","",1484564634508029],
>["value2:646464","",1484564634508029]]}
> ]
> # Upgrade node2,3
> $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm 
> node${i} start;ccm node${i} nodetool upgradesstables; done
> # After upgrade node2,3, the row inserted from node1 is splitting in node2,3
> $ ccm node2 cqlsh -e "SELECT * FROM test.test ;"  
>   
>  id  | value1 | value2
> -++
>  aaa |   null |   null
>  aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'}
>  bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'}
> (3 rows)
> # Results of sstabledump
> # node1
> [
>   {
> "partition" : {
>   "key" : [ "aaa" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 17,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" },
> "cells" : [
>   { "name" : "value1", "deletion_info" : { "marked_deleted" : 
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } 
> },
>   { "name" : "value1", "path" : [ "aaa" ], "value" : "" },
>   { "name" : "value1", "path" : [ "bbb" ], "value" : "" },
>   { "name" : "value2", "deletion_info" : { "marked_deleted" : 
> "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } 
> },
>   { "name" : "value2", "path" : [ "ccc" ], "value" : "" },
>   { "name" : "value2", "path" : [ "ddd" ], "value" : "" }
> ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "bbb" ],
>   "position" : 48
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 65,
> "liveness_info" : { "tstamp" : "2017-01-16T11:03:54.508029Z" },
> "cells" : [
>   { "name" : "value1", "deletion_info" : { 

[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831782#comment-15831782
 ] 

Jason Brown commented on CASSANDRA-4663:


Yeah, I think you are correct here. I didn't look that far up the stack (into 
{{RangeStreamer}}), but that will surely parallelize transfers by keyspace.

I've kicked off the tests here:
||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/4663-trunk]|
|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-4663-trunk-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-4663-trunk-testall/]|

If everything looks good with the tests, I'll rename the ticket to something 
more appropriate and commit.

> Streaming sends one file at a time serially. 
> -
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 
> 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch
>
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12202) LongLeveledCompactionStrategyTest flapping in 2.1, 2.2, 3.0

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831737#comment-15831737
 ] 

Jason Brown commented on CASSANDRA-12202:
-

[~yukim] Did this ever get committed?

> LongLeveledCompactionStrategyTest flapping in 2.1, 2.2, 3.0
> ---
>
> Key: CASSANDRA-12202
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12202
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> We actually fixed this for 3.7+ in CASSANDRA-11657, need to backport that fix 
> to 2.1+



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13103) incorrect jvm metric names

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13103:

Assignee: Alain Rastoul

> incorrect jvm metric names
> --
>
> Key: CASSANDRA-13103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13103
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Alain Rastoul
>Assignee: Alain Rastoul
>Priority: Minor
>  Labels: lhf
> Fix For: 3.9, 3.x
>
> Attachments: Fix-incorrect-jvm-metric-names-CASSANDRA-13103.patch
>
>
> Some jvm metrics have a double dot in name like:
> jvm.memory..total.max , jvm.memory..total.init (etc).
> it seems that an extra dot is added at the end of the name in 
> CassandraDaemon.java, around line 367 (in 3.0.10):
> ...
> // enable metrics provided by metrics-jvm.jar
> CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new 
> BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
> CassandraMetricsRegistry.Metrics.register("jvm.gc.", new 
> GarbageCollectorMetricSet());
> CassandraMetricsRegistry.Metrics.register("jvm.memory.", new 
> MemoryUsageGaugeSet());
> and also added in append method of MetricRegistry.
> Call stack is:
> MetricRegistry>>registerAll(String prefix, MetricSet metrics)
> MetricRegistry>>static String name(String name, String... names)
> MetricRegistry>>static void append(StringBuilder builder, String part)
> and in append the dot is also added:
> ...
> if(builder.length() > 0) {
> builder.append('.');
> }
> builder.append(part);
> ...
> The codahale MetricRegistry class seems to have no recent modification of 
> name or append methods, so it look like a small bug.
> May be the fix could be to simply not to add  the final dot in the metric 
> name, ie  "jvm.buffers"  instead of "jvm.buffers."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12850) Add the duration type to the protocol V5

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831687#comment-15831687
 ] 

Benjamin Lerer commented on CASSANDRA-12850:


[~thobbs] could you review?

> Add the duration type to the protocol V5
> 
>
> Key: CASSANDRA-12850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12850
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The Duration type need to be added to the protocol specifications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12850) Add the duration type to the protocol V5

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-12850:
---
Status: Patch Available  (was: In Progress)

> Add the duration type to the protocol V5
> 
>
> Key: CASSANDRA-12850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12850
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The Duration type need to be added to the protocol specifications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12850) Add the duration type to the protocol V5

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831685#comment-15831685
 ] 

Benjamin Lerer commented on CASSANDRA-12850:


||[3.11|https://github.com/apache/cassandra/compare/trunk...blerer:12850-3.11]|[utest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-3.11-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-3.11-dtest/]|
||[trunk|https://github.com/apache/cassandra/compare/trunk...blerer:12850-trunk]|[utest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-trunk-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-trunk-dtest/]|

The patch add {{Duration}} to the protocol specification and to the 
{{DataType}}.
While running the tests, I faced an {{IllegalStateException}} in 
{{OptionCodec}} caused by the fact that 2 {{DataType}}s where having the same 
id (the one of {{CUSTOM}}.
As {{OptionCodec}} was only used by {{DataType}} I simplified it and converted 
it into an inner class. That change helped to fix the problem as the 
{{protocolVersion}} of the {{DataType}} was needed. 




> Add the duration type to the protocol V5
> 
>
> Key: CASSANDRA-12850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12850
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The Duration type need to be added to the protocol specifications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13138) SASI tries to fetch an extra page when resultset size is same size as page size

[Alex Petrov (JIRA)]

Alex Petrov created CASSANDRA-13138:
---

 Summary: SASI tries to fetch an extra page when resultset size is 
same size as page size
 Key: CASSANDRA-13138
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13138
 Project: Cassandra
  Issue Type: Bug
Reporter: Alex Petrov


For example, in a dataset that would return 10 rows, SASI would try (and return 
an empty page) to fetch the next page, while filtering and 2i will return 
results correctly:

{code}
 pk | ck1 | ck2 | reg1 | reg2 | reg3
+-+-+--+--+--
  6 |   5 |   5 |5 |5 |   10
  7 |   5 |   5 |5 |5 |   10
  9 |   5 |   5 |5 |5 |   10
  4 |   5 |   5 |5 |5 |   10
  3 |   5 |   5 |5 |5 |   10
  5 |   5 |   5 |5 |5 |   10
  0 |   5 |   5 |5 |5 |   10
  8 |   5 |   5 |5 |5 |   10
  2 |   5 |   5 |5 |5 |   10
  1 |   5 |   5 |5 |5 |   10

---MORE---
(10 rows)
{code}
(that {{--MORE--}} shouldn't have been there) 

This might be an inherent limitation, although even if it is we can opt out for 
fetching limit+1 if the data limits aren't exhausted. Although it seems that 
there should be a solution for it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)