cassandra git commit: fix typo introduced in #12142, regarding version constants

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk f531c67d6 -> 0e23d175a


fix typo introduced in #12142, regarding version constants


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0e23d175
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0e23d175
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0e23d175

Branch: refs/heads/trunk
Commit: 0e23d175ae0538e2360d1f18f30619d537d82ed2
Parents: f531c67
Author: Dave Brosius 
Authored: Wed Aug 3 23:47:12 2016 -0400
Committer: Dave Brosius 
Committed: Wed Aug 3 23:47:12 2016 -0400

--
 src/java/org/apache/cassandra/transport/Event.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e23d175/src/java/org/apache/cassandra/transport/Event.java
--
diff --git a/src/java/org/apache/cassandra/transport/Event.java 
b/src/java/org/apache/cassandra/transport/Event.java
index 232d817..618d722 100644
--- a/src/java/org/apache/cassandra/transport/Event.java
+++ b/src/java/org/apache/cassandra/transport/Event.java
@@ -345,7 +345,7 @@ public abstract class Event
 {
 if (target == Target.FUNCTION || target == Target.AGGREGATE)
 {
-if (version >= Server.VERSION_3)
+if (version >= Server.VERSION_4)
 return CBUtil.sizeOfEnumValue(change)
+ CBUtil.sizeOfEnumValue(target)
+ CBUtil.sizeOfString(keyspace)

cassandra git commit: avoid unlikely race condition wrt disk sizing, producing a seemingly bogus exception message

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk f97e41049 -> f531c67d6


avoid unlikely race condition wrt disk sizing, producing a seemingly bogus 
exception message


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f531c67d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f531c67d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f531c67d

Branch: refs/heads/trunk
Commit: f531c67d687a004bcda57582e68f90b1901e992d
Parents: f97e410
Author: Dave Brosius 
Authored: Wed Aug 3 23:25:46 2016 -0400
Committer: Dave Brosius 
Committed: Wed Aug 3 23:25:46 2016 -0400

--
 .../db/compaction/writers/CompactionAwareWriter.java | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f531c67d/src/java/org/apache/cassandra/db/compaction/writers/CompactionAwareWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/writers/CompactionAwareWriter.java
 
b/src/java/org/apache/cassandra/db/compaction/writers/CompactionAwareWriter.java
index 3557022..205aebe 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/writers/CompactionAwareWriter.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/writers/CompactionAwareWriter.java
@@ -208,16 +208,20 @@ public abstract class CompactionAwareWriter extends 
Transactional.AbstractTransa
 if (directory == null)
 directory = sstable.descriptor.directory;
 if (!directory.equals(sstable.descriptor.directory))
+{
 logger.trace("All sstables not from the same disk - putting 
results in {}", directory);
+break;
+}
 }
 Directories.DataDirectory d = 
getDirectories().getDataDirectoryForFile(directory);
 if (d != null)
 {
-if (d.getAvailableSpace() < estimatedWriteSize)
+long availableSpace = d.getAvailableSpace();
+if (availableSpace < estimatedWriteSize)
 throw new RuntimeException(String.format("Not enough space to 
write %s to %s (%s available)",
  
FBUtilities.prettyPrintMemory(estimatedWriteSize),
  d.location,
- 
FBUtilities.prettyPrintMemory(d.getAvailableSpace(;
+ 
FBUtilities.prettyPrintMemory(availableSpace)));
 logger.trace("putting compaction results in {}", directory);
 return d;
 }

[jira] [Updated] (CASSANDRA-11031) MultiTenant : support “ALLOW FILTERING" for Partition Key

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-11031:
-
Status: Patch Available  (was: Open)

> MultiTenant : support “ALLOW FILTERING" for Partition Key
> -
>
> Key: CASSANDRA-11031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11031
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Minor
> Fix For: 3.x
>
> Attachments: CASSANDRA-11031-3.7.patch
>
>
> Currently, Allow Filtering only works for secondary Index column or 
> clustering columns. And it's slow, because Cassandra will read all data from 
> SSTABLE from hard-disk to memory to filter.
> But we can support allow filtering on Partition Key, as far as I know, 
> Partition Key is in memory, so we can easily filter them, and then read 
> required data from SSTable.
> This will similar to "Select * from table" which scan through entire cluster.
> CREATE TABLE multi_tenant_table (
>   tenant_id text,
>   pk2 text,
>   c1 text,
>   c2 text,
>   v1 text,
>   v2 text,
>   PRIMARY KEY ((tenant_id,pk2),c1,c2)
> ) ;
> Select * from multi_tenant_table where tenant_id = "datastax" allow filtering;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: simple reduction of 2n UUID.toString() calls

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7dccb3137 -> f97e41049


simple reduction of 2n UUID.toString() calls


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f97e4104
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f97e4104
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f97e4104

Branch: refs/heads/trunk
Commit: f97e410495d7bef19708ba088e56bea8f4c24206
Parents: 7dccb31
Author: Dave Brosius 
Authored: Wed Aug 3 22:50:51 2016 -0400
Committer: Dave Brosius 
Committed: Wed Aug 3 22:50:51 2016 -0400

--
 src/java/org/apache/cassandra/service/ActiveRepairService.java | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f97e4104/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java 
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index 1ef5dfe..435c7c8 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -389,10 +389,11 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
  */
 public synchronized ParentRepairSession removeParentRepairSession(UUID 
parentSessionId)
 {
+String snapshotName = parentSessionId.toString();
 for (ColumnFamilyStore cfs : 
getParentRepairSession(parentSessionId).columnFamilyStores.values())
 {
-if (cfs.snapshotExists(parentSessionId.toString()))
-cfs.clearSnapshot(parentSessionId.toString());
+if (cfs.snapshotExists(snapshotName))
+cfs.clearSnapshot(snapshotName);
 }
 return parentRepairSessions.remove(parentSessionId);
 }

cassandra git commit: simple boxing fixes: parseXXX for primitives, valueOf for boxed objects

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk d4410263d -> 7dccb3137


simple boxing fixes: parseXXX for primitives, valueOf for boxed objects


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7dccb313
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7dccb313
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7dccb313

Branch: refs/heads/trunk
Commit: 7dccb3137375ce310a5827ffd493dbde93ad147a
Parents: d441026
Author: Dave Brosius 
Authored: Wed Aug 3 22:26:02 2016 -0400
Committer: Dave Brosius 
Committed: Wed Aug 3 22:26:02 2016 -0400

--
 .../org/apache/cassandra/cql3/statements/TableAttributes.java  | 2 +-
 src/java/org/apache/cassandra/index/sasi/conf/IndexMode.java   | 6 +++---
 src/java/org/apache/cassandra/io/compress/LZ4Compressor.java   | 2 +-
 .../org/apache/cassandra/io/sstable/format/SSTableFormat.java  | 2 +-
 src/java/org/apache/cassandra/net/OutboundTcpConnection.java   | 2 +-
 src/java/org/apache/cassandra/schema/SchemaKeyspace.java   | 2 +-
 .../org/apache/cassandra/service/NativeTransportService.java   | 2 +-
 src/java/org/apache/cassandra/service/StorageService.java  | 2 +-
 .../org/apache/cassandra/tools/nodetool/TopPartitions.java | 4 ++--
 src/java/org/apache/cassandra/tracing/TraceStateImpl.java  | 2 +-
 src/java/org/apache/cassandra/utils/HeapUtils.java | 2 +-
 11 files changed, 14 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dccb313/src/java/org/apache/cassandra/cql3/statements/TableAttributes.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/TableAttributes.java 
b/src/java/org/apache/cassandra/cql3/statements/TableAttributes.java
index dee3385..63e1eec 100644
--- a/src/java/org/apache/cassandra/cql3/statements/TableAttributes.java
+++ b/src/java/org/apache/cassandra/cql3/statements/TableAttributes.java
@@ -142,7 +142,7 @@ public final class TableAttributes extends 
PropertyDefinitions
 String value = 
compressionOpts.get(Option.CRC_CHECK_CHANCE.toString().toLowerCase());
 try
 {
-return Double.parseDouble(value);
+return Double.valueOf(value);
 }
 catch (NumberFormatException e)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dccb313/src/java/org/apache/cassandra/index/sasi/conf/IndexMode.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/conf/IndexMode.java 
b/src/java/org/apache/cassandra/index/sasi/conf/IndexMode.java
index 1c85ed5..c66dd02 100644
--- a/src/java/org/apache/cassandra/index/sasi/conf/IndexMode.java
+++ b/src/java/org/apache/cassandra/index/sasi/conf/IndexMode.java
@@ -141,11 +141,11 @@ public class IndexMode
 {
 analyzerClass = 
Class.forName(indexOptions.get(INDEX_ANALYZER_CLASS_OPTION));
 isAnalyzed = indexOptions.get(INDEX_ANALYZED_OPTION) == null
-  ? true : 
Boolean.valueOf(indexOptions.get(INDEX_ANALYZED_OPTION));
+  ? true : 
Boolean.parseBoolean(indexOptions.get(INDEX_ANALYZED_OPTION));
 }
 else if (indexOptions.get(INDEX_ANALYZED_OPTION) != null)
 {
-isAnalyzed = 
Boolean.valueOf(indexOptions.get(INDEX_ANALYZED_OPTION));
+isAnalyzed = 
Boolean.parseBoolean(indexOptions.get(INDEX_ANALYZED_OPTION));
 }
 }
 catch (ClassNotFoundException e)
@@ -163,7 +163,7 @@ public class IndexMode
 
 isLiteral = literalOption == null
 ? (validator instanceof UTF8Type || validator 
instanceof AsciiType)
-: Boolean.valueOf(literalOption);
+: Boolean.parseBoolean(literalOption);
 }
 catch (Exception e)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dccb313/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java
--
diff --git a/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java 
b/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java
index 3fd889e..1b3844d 100644
--- a/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java
+++ b/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java
@@ -207,7 +207,7 @@ public class LZ4Compressor implements ICompressor
 Integer level;
 try
 {
-level = Integer.parseInt(compressionLevel);
+level = Integer.valueOf(compressionLevel);
 }
 catch 

[jira] [Updated] (CASSANDRA-12377) Add byteman support for 2.2

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-12377:
---
Status: Patch Available  (was: Open)

Patch here: https://github.com/yukim/cassandra/tree/byteman_support_2.2

> Add byteman support for 2.2
> ---
>
> Key: CASSANDRA-12377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12377
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Trivial
> Fix For: 2.2.x
>
>
> Using byteman for dtest is useful to interrupt streaming reliably 
> (CASSANDRA-10810 / https://github.com/riptano/cassandra-dtest/pull/1145).
> Unfortunately, it is available for 3.0+.
> This ticket backports it to 2.2.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12377) Add byteman support for 2.2

[Yuki Morishita (JIRA)]

Yuki Morishita created CASSANDRA-12377:
--

 Summary: Add byteman support for 2.2
 Key: CASSANDRA-12377
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12377
 Project: Cassandra
  Issue Type: Test
  Components: Testing
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Trivial
 Fix For: 2.2.x


Using byteman for dtest is useful to interrupt streaming reliably 
(CASSANDRA-10810 / https://github.com/riptano/cassandra-dtest/pull/1145).

Unfortunately, it is available for 3.0+.

This ticket backports it to 2.2.x.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11345) Assertion Errors "Memory was freed" during streaming

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406966#comment-15406966
 ] 

Yuki Morishita commented on CASSANDRA-11345:


3.9/trunk branch do not compile newly added unit test. (see github comment.)
I wonder why tests were running on CI though.

> Assertion Errors "Memory was freed" during streaming
> 
>
> Key: CASSANDRA-11345
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11345
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Jean-Francois Gosselin
>Assignee: Paulo Motta
>
> We encountered the following AssertionError (twice on the same node) during a 
> repair :
> On node /172.16.63.41
> {noformat}
> INFO  [STREAM-IN-/10.174.216.160] 2016-03-09 02:38:13,900 
> StreamResultFuture.java:180 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Session with /10.174.216.160 is complete  
>   
> WARN  [STREAM-IN-/10.174.216.160] 2016-03-09 02:38:13,900 
> StreamResultFuture.java:207 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Stream failed   
> ERROR [STREAM-OUT-/10.174.216.160] 2016-03-09 02:38:13,906 
> StreamSession.java:505 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Streaming error occurred
> java.lang.AssertionError: Memory was freed
>   
>
> at 
> org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) 
> ~[apache-cassandra-2.1.13.jar:2.1.13] 
>   
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:249) 
> ~[apache-cassandra-2.1.13.jar:2.1.13] 
>  
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.getTotalSizeForSections(CompressionMetadata.java:247)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]
> at 
> org.apache.cassandra.streaming.messages.FileMessageHeader.size(FileMessageHeader.java:112)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]
> at 
> org.apache.cassandra.streaming.StreamSession.fileSent(StreamSession.java:546) 
> ~[apache-cassandra-2.1.13.jar:2.1.13] 
> 
> at 
> org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:50)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]  
> at 
> org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:41)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]  
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:45)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:351)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]   
> at 
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:331)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]   
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
>   
>
> {noformat} 
> On node /10.174.216.160
>  
> {noformat}   
> ERROR [STREAM-OUT-/172.16.63.41] 2016-03-09 02:38:14,140 
> StreamSession.java:505 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Streaming error occurred  
> java.io.IOException: Connection reset by peer 
>   
>
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[na:1.7.0_65] 
>   
>
> at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) 
> ~[na:1.7.0_65]
>   
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) 
> ~[na:1.7.0_65]
>   
> at sun.nio.ch.IOUtil.write(IOUtil.java:65) ~[na:1.7.0_65] 
>   
>
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487) 
> 

[jira] [Updated] (CASSANDRA-10643) Implement compaction for a specific token range

[Vishy Kasar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vishy Kasar updated CASSANDRA-10643:

Status: Patch Available  (was: Awaiting Feedback)

> Implement compaction for a specific token range
> ---
>
> Key: CASSANDRA-10643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Vishy Kasar
>Assignee: Vishy Kasar
>  Labels: lcs
> Attachments: 10643-trunk-REV01.txt, 10643-trunk-REV02.txt, 
> 10643-trunk-REV03.txt
>
>
> We see repeated cases in production (using LCS) where small number of users 
> generate a large number repeated updates or tombstones. Reading data of such 
> users brings in large amounts of data in to java process. Apart from the read 
> itself being slow for the user, the excessive GC affects other users as well. 
> Our solution so far is to move from LCS to SCS and back. This takes long and 
> is an over kill if the number of outliers is small. For such cases, we can 
> implement the point compaction of a token range. We make the nodetool compact 
> take a starting and ending token range and compact all the SSTables that fall 
> with in that range. We can refuse to compact if the number of sstables is 
> beyond a max_limit.
> Example: 
> nodetool -st 3948291562518219268 -et 3948291562518219269 compact keyspace 
> table



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12311) Propagate TombstoneOverwhelmingException to the client

[Geoffrey Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406949#comment-15406949
 ] 

Geoffrey Yu edited comment on CASSANDRA-12311 at 8/4/16 1:21 AM:
-

Those ideas sound good to me. I can see how having extensibility in the failure 
codes can be useful so that we don't need to wait for protocol version bumps. 
Also passing back an endpoint to failure code map would be nice since we won't 
need to interpret the potentially different responses from the replicas at the 
coordinator to determine which (single) failure code should be used.

I attached a patch with those changes incorporated. Since we need to pass some 
sort of failure code back from the replicas, I wanted to use the same set of 
failure codes between nodes as between the client and coordinator. So I placed 
the codes in a new enum {{RequestFailureReason}} and placed the map under 
{{RequestFailureException}}, meaning {{WriteFailureException}} s will carry 
this endpoint to failure code map as well. Please let me know what you think.


was (Author: geoffxy):
Those ideas sound good to me. I can see how having extensibility in the failure 
codes can be useful so that we don't need to wait for protocol version bumps. 
Also passing back an endpoint to failure code map would be nice since we won't 
need to interpret the potentially different responses from the replicas at the 
coordinator to determine which (single) failure code should be used.

I attached a patch with those changes incorporated. Since we need to pass some 
sort of failure code back from the replicas, I wanted to use the same set of 
failure codes between nodes as between the client and coordinator. So I placed 
the codes in a new enum {{RequestFailureReason}} and placed the map under 
{{RequestFailureException}}, meaning {{WriteFailureException}}s will carry this 
endpoint to failure code map as well. Please let me know what you think.

> Propagate TombstoneOverwhelmingException to the client
> --
>
> Key: CASSANDRA-12311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12311
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
>  Labels: client-impacting, doc-impacting
> Fix For: 4.x
>
> Attachments: 12311-trunk-v2.txt, 12311-trunk-v3.txt, 12311-trunk.txt
>
>
> Right now if a data node fails to perform a read because it ran into a 
> {{TombstoneOverwhelmingException}}, it only responds back to the coordinator 
> node with a generic failure. Under this scheme, the coordinator won't be able 
> to know exactly why the request failed and subsequently the client only gets 
> a generic {{ReadFailureException}}. It would be useful to inform the client 
> that their read failed because we read too many tombstones. We should have 
> the data nodes reply with a failure type so the coordinator can pass this 
> information to the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12311) Propagate TombstoneOverwhelmingException to the client

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-12311:

Attachment: 12311-trunk-v3.txt

Those ideas sound good to me. I can see how having extensibility in the failure 
codes can be useful so that we don't need to wait for protocol version bumps. 
Also passing back an endpoint to failure code map would be nice since we won't 
need to interpret the potentially different responses from the replicas at the 
coordinator to determine which (single) failure code should be used.

I attached a patch with those changes incorporated. Since we need to pass some 
sort of failure code back from the replicas, I wanted to use the same set of 
failure codes between nodes as between the client and coordinator. So I placed 
the codes in a new enum {{RequestFailureReason}} and placed the map under 
{{RequestFailureException}}, meaning {{WriteFailureException}}s will carry this 
endpoint to failure code map as well. Please let me know what you think.

> Propagate TombstoneOverwhelmingException to the client
> --
>
> Key: CASSANDRA-12311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12311
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
>  Labels: client-impacting, doc-impacting
> Fix For: 4.x
>
> Attachments: 12311-trunk-v2.txt, 12311-trunk-v3.txt, 12311-trunk.txt
>
>
> Right now if a data node fails to perform a read because it ran into a 
> {{TombstoneOverwhelmingException}}, it only responds back to the coordinator 
> node with a generic failure. Under this scheme, the coordinator won't be able 
> to know exactly why the request failed and subsequently the client only gets 
> a generic {{ReadFailureException}}. It would be useful to inform the client 
> that their read failed because we read too many tombstones. We should have 
> the data nodes reply with a failure type so the coordinator can pass this 
> information to the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10643) Implement compaction for a specific token range

[Vishy Kasar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406930#comment-15406930
 ] 

Vishy Kasar commented on CASSANDRA-10643:
-

Thanks for the quick review Marcus. I have taken care of most of your feedback.

* Ongoing compactions are cancelled
* In CompactionManager, submitOnSSTables and submitTask were merged in to a 
single method. I have kept the submitOnSSTables public to keep parity with 
submitMaximal. 
* In ColumnFamilyStore#sstablesInBounds:
  ** Used the Set
  ** Used the View.sstablesInBounds 
  ** I have kept the Range to keep parity with Repair

> Implement compaction for a specific token range
> ---
>
> Key: CASSANDRA-10643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Vishy Kasar
>Assignee: Vishy Kasar
>  Labels: lcs
> Attachments: 10643-trunk-REV01.txt, 10643-trunk-REV02.txt, 
> 10643-trunk-REV03.txt
>
>
> We see repeated cases in production (using LCS) where small number of users 
> generate a large number repeated updates or tombstones. Reading data of such 
> users brings in large amounts of data in to java process. Apart from the read 
> itself being slow for the user, the excessive GC affects other users as well. 
> Our solution so far is to move from LCS to SCS and back. This takes long and 
> is an over kill if the number of outliers is small. For such cases, we can 
> implement the point compaction of a token range. We make the nodetool compact 
> take a starting and ending token range and compact all the SSTables that fall 
> with in that range. We can refuse to compact if the number of sstables is 
> beyond a max_limit.
> Example: 
> nodetool -st 3948291562518219268 -et 3948291562518219269 compact keyspace 
> table



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406928#comment-15406928
 ] 

Pavel Yaskevich commented on CASSANDRA-12374:
-

This is definitely a porting bug, I'll be happy to review your changes, 
[~ifesdjeen].

> Can't rebuild SASI index
> 
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Petrov
>
> There's been no real requirement for that so far. 
> As [~beobal] has pointed out, it's not a big issue, since that only could be 
> needed when index files are lost, data corruption on disk (hardware issue) 
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with 
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}} 
> is called. However, since SASI keeps track of SSTables for the index, it's 
> going to filter them out with {{.filter((sstable) -> 
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data) 
> right before we re-index them. This is also a blocker for [CASSANDRA-11990] 
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it 
> released earlier and for all branches affected.
> cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11823) Creating a table leads to a race with GraphiteReporter

[Edward Ribeiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406925#comment-15406925
 ] 

Edward Ribeiro commented on CASSANDRA-11823:


Cool! Really glad to help. Thanks for the opportunity. :)

> Creating a table leads to a race with GraphiteReporter
> --
>
> Key: CASSANDRA-11823
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11823
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Stefano Ortolani
>Assignee: Edward Ribeiro
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-11823.patch
>
>
> Happened only on 3/4 nodes out of 13.
> {code:xml}
> INFO  [MigrationStage:1] 2016-05-18 00:34:11,566 ColumnFamilyStore.java:381 - 
> Initializing schema.table
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-18 00:34:11,569 
> ScheduledReporter.java:119 - RuntimeException thrown from 
> GraphiteReporter#report. Exception was suppressed.
> java.util.ConcurrentModificationException: null
>   at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) 
> ~[na:1.8.0_91]
>   at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_91]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:690) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_91]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11823) Creating a table leads to a race with GraphiteReporter

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11823:
-
Component/s: Observability

> Creating a table leads to a race with GraphiteReporter
> --
>
> Key: CASSANDRA-11823
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11823
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Stefano Ortolani
>Assignee: Edward Ribeiro
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-11823.patch
>
>
> Happened only on 3/4 nodes out of 13.
> {code:xml}
> INFO  [MigrationStage:1] 2016-05-18 00:34:11,566 ColumnFamilyStore.java:381 - 
> Initializing schema.table
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-18 00:34:11,569 
> ScheduledReporter.java:119 - RuntimeException thrown from 
> GraphiteReporter#report. Exception was suppressed.
> java.util.ConcurrentModificationException: null
>   at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) 
> ~[na:1.8.0_91]
>   at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_91]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:690) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_91]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11823) Creating a table leads to a race with GraphiteReporter

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11823:
-
   Resolution: Fixed
Fix Version/s: 3.9
   3.0.9
   Status: Resolved  (was: Patch Available)

> Creating a table leads to a race with GraphiteReporter
> --
>
> Key: CASSANDRA-11823
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11823
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Stefano Ortolani
>Assignee: Edward Ribeiro
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-11823.patch
>
>
> Happened only on 3/4 nodes out of 13.
> {code:xml}
> INFO  [MigrationStage:1] 2016-05-18 00:34:11,566 ColumnFamilyStore.java:381 - 
> Initializing schema.table
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-18 00:34:11,569 
> ScheduledReporter.java:119 - RuntimeException thrown from 
> GraphiteReporter#report. Exception was suppressed.
> java.util.ConcurrentModificationException: null
>   at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) 
> ~[na:1.8.0_91]
>   at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_91]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:690) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_91]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11823) Creating a table leads to a race with GraphiteReporter

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406919#comment-15406919
 ] 

Stefania commented on CASSANDRA-11823:
--

CI looks good, committed to 3.0 as 52be7bac4160c392605f2ec6c2f98d4e49ccf2fe and 
merged upwards.

> Creating a table leads to a race with GraphiteReporter
> --
>
> Key: CASSANDRA-11823
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11823
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Stefano Ortolani
>Assignee: Edward Ribeiro
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-11823.patch
>
>
> Happened only on 3/4 nodes out of 13.
> {code:xml}
> INFO  [MigrationStage:1] 2016-05-18 00:34:11,566 ColumnFamilyStore.java:381 - 
> Initializing schema.table
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-18 00:34:11,569 
> ScheduledReporter.java:119 - RuntimeException thrown from 
> GraphiteReporter#report. Exception was suppressed.
> java.util.ConcurrentModificationException: null
>   at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) 
> ~[na:1.8.0_91]
>   at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_91]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:690) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) 
> ~[metrics-core-3.1.0.jar:3.1.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_91]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10643) Implement compaction for a specific token range

[Vishy Kasar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vishy Kasar updated CASSANDRA-10643:

Attachment: 10643-trunk-REV03.txt

> Implement compaction for a specific token range
> ---
>
> Key: CASSANDRA-10643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Vishy Kasar
>Assignee: Vishy Kasar
>  Labels: lcs
> Attachments: 10643-trunk-REV01.txt, 10643-trunk-REV02.txt, 
> 10643-trunk-REV03.txt
>
>
> We see repeated cases in production (using LCS) where small number of users 
> generate a large number repeated updates or tombstones. Reading data of such 
> users brings in large amounts of data in to java process. Apart from the read 
> itself being slow for the user, the excessive GC affects other users as well. 
> Our solution so far is to move from LCS to SCS and back. This takes long and 
> is an over kill if the number of outliers is small. For such cases, we can 
> implement the point compaction of a token range. We make the nodetool compact 
> take a starting and ending token range and compact all the SSTables that fall 
> with in that range. We can refuse to compact if the number of sstables is 
> beyond a max_limit.
> Example: 
> nodetool -st 3948291562518219268 -et 3948291562518219269 compact keyspace 
> table



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[6/6] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[stefania]

Merge branch 'cassandra-3.9' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d4410263
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d4410263
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d4410263

Branch: refs/heads/trunk
Commit: d4410263db587b4250fb1a9b4bc73d14c52f4fc0
Parents: 4e21f72 042e1f7
Author: Stefania Alborghetti 
Authored: Thu Aug 4 08:51:14 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Aug 4 08:51:14 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/metrics/TableMetrics.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4410263/CHANGES.txt
--

[1/6] cassandra git commit: Fixed ConcurrentModificationException when reading metrics in GraphiteReporter

[stefania]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 3431934b4 -> 52be7bac4
  refs/heads/cassandra-3.9 b603720e4 -> 042e1f76a
  refs/heads/trunk 4e21f7267 -> d4410263d


Fixed ConcurrentModificationException when reading metrics in GraphiteReporter

patch by Edward Ribeiro; reviewed by Stefania Alborghetti for CASSANDRA-11823


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52be7bac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52be7bac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52be7bac

Branch: refs/heads/cassandra-3.0
Commit: 52be7bac4160c392605f2ec6c2f98d4e49ccf2fe
Parents: 3431934
Author: Edward Ribeiro 
Authored: Wed Aug 3 10:07:13 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Aug 4 08:47:49 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/metrics/TableMetrics.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52be7bac/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8b8abb..f0ceb70 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.9
+ * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
  * Fix upgrade of super columns on thrift (CASSANDRA-12335)
  * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
  * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/52be7bac/src/java/org/apache/cassandra/metrics/TableMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/TableMetrics.java 
b/src/java/org/apache/cassandra/metrics/TableMetrics.java
index 85bf7f6..a21e8ca 100644
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.metrics;
 import java.nio.ByteBuffer;
 import java.util.*;
 import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
 
 import com.codahale.metrics.*;
@@ -788,7 +789,7 @@ public class TableMetrics
  */
 private boolean register(String name, String alias, Metric metric)
 {
-boolean ret = allTableMetrics.putIfAbsent(name,  new HashSet<>()) == 
null;
+boolean ret = allTableMetrics.putIfAbsent(name, 
ConcurrentHashMap.newKeySet()) == null;
 allTableMetrics.get(name).add(metric);
 all.put(name, alias);
 return ret;

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/042e1f76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/042e1f76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/042e1f76

Branch: refs/heads/trunk
Commit: 042e1f76a993bb8f14ce31a13bde421ed071
Parents: b603720 52be7ba
Author: Stefania Alborghetti 
Authored: Thu Aug 4 08:50:04 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Aug 4 08:50:04 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/metrics/TableMetrics.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/042e1f76/CHANGES.txt
--
diff --cc CHANGES.txt
index 62a6e6f,f0ceb70..31d9434
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,9 -1,5 +1,10 @@@
 -3.0.9
 +3.9
 + * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * cqlsh: Fix handling of $$-escaped strings (CASSANDRA-12189)
 + * Fix SSL JMX requiring truststore containing server cert (CASSANDRA-12109)
 +Merged from 3.0:
+  * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
   * Fix upgrade of super columns on thrift (CASSANDRA-12335)
   * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
   * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/042e1f76/src/java/org/apache/cassandra/metrics/TableMetrics.java
--

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/042e1f76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/042e1f76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/042e1f76

Branch: refs/heads/cassandra-3.9
Commit: 042e1f76a993bb8f14ce31a13bde421ed071
Parents: b603720 52be7ba
Author: Stefania Alborghetti 
Authored: Thu Aug 4 08:50:04 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Aug 4 08:50:04 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/metrics/TableMetrics.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/042e1f76/CHANGES.txt
--
diff --cc CHANGES.txt
index 62a6e6f,f0ceb70..31d9434
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,9 -1,5 +1,10 @@@
 -3.0.9
 +3.9
 + * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * cqlsh: Fix handling of $$-escaped strings (CASSANDRA-12189)
 + * Fix SSL JMX requiring truststore containing server cert (CASSANDRA-12109)
 +Merged from 3.0:
+  * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
   * Fix upgrade of super columns on thrift (CASSANDRA-12335)
   * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
   * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/042e1f76/src/java/org/apache/cassandra/metrics/TableMetrics.java
--

[3/6] cassandra git commit: Fixed ConcurrentModificationException when reading metrics in GraphiteReporter

[stefania]

Fixed ConcurrentModificationException when reading metrics in GraphiteReporter

patch by Edward Ribeiro; reviewed by Stefania Alborghetti for CASSANDRA-11823


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52be7bac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52be7bac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52be7bac

Branch: refs/heads/trunk
Commit: 52be7bac4160c392605f2ec6c2f98d4e49ccf2fe
Parents: 3431934
Author: Edward Ribeiro 
Authored: Wed Aug 3 10:07:13 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Aug 4 08:47:49 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/metrics/TableMetrics.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52be7bac/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8b8abb..f0ceb70 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.9
+ * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
  * Fix upgrade of super columns on thrift (CASSANDRA-12335)
  * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
  * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/52be7bac/src/java/org/apache/cassandra/metrics/TableMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/TableMetrics.java 
b/src/java/org/apache/cassandra/metrics/TableMetrics.java
index 85bf7f6..a21e8ca 100644
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.metrics;
 import java.nio.ByteBuffer;
 import java.util.*;
 import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
 
 import com.codahale.metrics.*;
@@ -788,7 +789,7 @@ public class TableMetrics
  */
 private boolean register(String name, String alias, Metric metric)
 {
-boolean ret = allTableMetrics.putIfAbsent(name,  new HashSet<>()) == 
null;
+boolean ret = allTableMetrics.putIfAbsent(name, 
ConcurrentHashMap.newKeySet()) == null;
 allTableMetrics.get(name).add(metric);
 all.put(name, alias);
 return ret;

[2/6] cassandra git commit: Fixed ConcurrentModificationException when reading metrics in GraphiteReporter

[stefania]

Fixed ConcurrentModificationException when reading metrics in GraphiteReporter

patch by Edward Ribeiro; reviewed by Stefania Alborghetti for CASSANDRA-11823


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52be7bac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52be7bac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52be7bac

Branch: refs/heads/cassandra-3.9
Commit: 52be7bac4160c392605f2ec6c2f98d4e49ccf2fe
Parents: 3431934
Author: Edward Ribeiro 
Authored: Wed Aug 3 10:07:13 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Aug 4 08:47:49 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/metrics/TableMetrics.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52be7bac/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e8b8abb..f0ceb70 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.9
+ * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
  * Fix upgrade of super columns on thrift (CASSANDRA-12335)
  * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
  * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/52be7bac/src/java/org/apache/cassandra/metrics/TableMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/TableMetrics.java 
b/src/java/org/apache/cassandra/metrics/TableMetrics.java
index 85bf7f6..a21e8ca 100644
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.metrics;
 import java.nio.ByteBuffer;
 import java.util.*;
 import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
 
 import com.codahale.metrics.*;
@@ -788,7 +789,7 @@ public class TableMetrics
  */
 private boolean register(String name, String alias, Metric metric)
 {
-boolean ret = allTableMetrics.putIfAbsent(name,  new HashSet<>()) == 
null;
+boolean ret = allTableMetrics.putIfAbsent(name, 
ConcurrentHashMap.newKeySet()) == null;
 allTableMetrics.get(name).add(metric);
 all.put(name, alias);
 return ret;

[jira] [Updated] (CASSANDRA-12101) DESCRIBE INDEX: missing quotes for case-sensitive index name

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12101:
-
Status: Open  (was: Awaiting Feedback)

> DESCRIBE INDEX: missing quotes for case-sensitive index name
> 
>
> Key: CASSANDRA-12101
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12101
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Julien
>Assignee: Stefania
>Priority: Minor
>  Labels: cqlsh, lhf
>
> Create a custom index with a case-sensitive name.
> The result of the DESCRIBE INDEX command does not have quotes around the 
> index name. As a result, the index cannot be recreated with this output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12101) DESCRIBE INDEX: missing quotes for case-sensitive index name

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406905#comment-15406905
 ] 

Stefania commented on CASSANDRA-12101:
--

Thanks, I'll keep this ticket in status OPEN until we can bundle into cqlsh a 
version of the driver that incorporates PYTHON-616.

> DESCRIBE INDEX: missing quotes for case-sensitive index name
> 
>
> Key: CASSANDRA-12101
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12101
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Julien
>Assignee: Stefania
>Priority: Minor
>  Labels: cqlsh, lhf
>
> Create a custom index with a case-sensitive name.
> The result of the DESCRIBE INDEX command does not have quotes around the 
> index name. As a result, the index cannot be recreated with this output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12176) dtest failure in materialized_views_test.TestMaterializedViews.complex_repair_test

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406901#comment-15406901
 ] 

Joshua McKenzie commented on CASSANDRA-12176:
-

ping [~mambocab]

> dtest failure in 
> materialized_views_test.TestMaterializedViews.complex_repair_test
> --
>
> Key: CASSANDRA-12176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12176
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Carl Yeksigian
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log, 
> node4.log, node4_debug.log, node4_gc.log, node5.log, node5_debug.log, 
> node5_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_novnode_dtest/8/testReport/materialized_views_test/TestMaterializedViews/complex_repair_test
> Failed on CassCI build cassandra-3.9_novnode_dtest #8
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/materialized_views_test.py", line 
> 956, in complex_repair_test
> session.execute("CREATE TABLE ks.t (id int PRIMARY KEY, v int, v2 text, 
> v3 decimal)"
>   File "cassandra/cluster.py", line 1941, in 
> cassandra.cluster.Session.execute (cassandra/cluster.c:33642)
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile).result()
>   File "cassandra/cluster.py", line 3629, in 
> cassandra.cluster.ResponseFuture.result (cassandra/cluster.c:69369)
> raise self._final_exception
> ' message="Keyspace ks doesn\'t exist">
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12264) dtest failure in repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406854#comment-15406854
 ] 

Paulo Motta commented on CASSANDRA-12264:
-

+1 after [multiplexer 
run|https://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/213/]
 looks good

> dtest failure in 
> repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test
> 
>
> Key: CASSANDRA-12264
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12264
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Marcus Eriksson
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/15/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test
> {code}
> Error Message
> 'Repaired at: 0' unexpectedly found in 'SSTable: 
> {code}
> Related failure:
> http://cassci.datastax.com/job/trunk_dtest/1315/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12323) dtest failure in compaction_test.TestCompaction_with_DateTieredCompactionStrategy.bloomfilter_size_test

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406824#comment-15406824
 ] 

Paulo Motta commented on CASSANDRA-12323:
-

test run failed, submitted another run as well as [100x multiplexer 
run|https://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/212/].

+1 after CI results look good.

> dtest failure in 
> compaction_test.TestCompaction_with_DateTieredCompactionStrategy.bloomfilter_size_test
> ---
>
> Key: CASSANDRA-12323
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12323
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Craig Kodman
>Assignee: Marcus Eriksson
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_novnode_dtest/19/testReport/compaction_test/TestCompaction_with_DateTieredCompactionStrategy/bloomfilter_size_test
> 500352 not less than or equal to 15



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9645) Make DateTieredCompactionStrategy easier to use

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406774#comment-15406774
 ] 

Jon Haddad commented on CASSANDRA-9645:
---

Seems like this can be closed, yes?

> Make DateTieredCompactionStrategy easier to use
> ---
>
> Key: CASSANDRA-9645
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9645
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>  Labels: dtcs
>
> It has proven to be quite difficult to use DTCS properly, we need to make it 
> easier and safer. Things to do could be (but there is surely more things) :
> * put warnings in logs if read repair is used
> * better debug logging which explains why certain sstables are selected etc
> * Auto-tune settings, this could be quite difficult, but we should atleast do 
> CASSANDRA-9130 to default max_sstable_age to gcgs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12358:

Reviewer: Branimir Lambov

> Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
> -
>
> Key: CASSANDRA-12358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12358
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.10
>
>
> 2i can be slow to flush for a variety of reasons. Potentially slower than the 
> rate at which Memtables can ingest and flush data. If this occurs the heap 
> fills up with Memtables that are waiting for PostFlush to run.
> This occurs because reclaiming the memory is done before PostFlush runs.
> I will post a branch that has the reclaim memory task run after PostFlush has 
> completed. As far as I can tell this is safe and correct since the memory is 
> committed up until that point.
> It's not clear to me if PostFlush has to bind the Memtables or not. I suspect 
> it does, but I'm not sure if that is a route I should go down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12343) Make 'static final boolean' easier to optimize for Hotspot

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12343:

Reviewer: Branimir Lambov

> Make 'static final boolean' easier to optimize for Hotspot
> --
>
> Key: CASSANDRA-12343
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12343
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Trivial
> Fix For: 3.x
>
>
> Hotspot is able to optimize condition checks on `static final` fields. But 
> the compiler can only optimize if the referenced "constant" is the first 
> condition to check. (If I understood the optimization in Hotspot correctly.)
> I.e. the first {{if}} block can be "eliminated" whereas the second cannot:
> {code}
> class Foo {
>   static final boolean CONST = /* some fragment evaluating to false */;
>   
>   public void doSomeStuff(boolean param) {
> if (!CONST) {
>   // this code block can be eliminated
> }
> if (!CONST && param) {
>   // this code block can be eliminated
> }
> if (param && !CONST) {
>   // this code block cannot be eliminated due to some compiler logic
> }
> 
>   }
> }
> {code}
> Linked patch changes the order in some {{if}} statements and migrates a few 
> methods to static final fields.
> ||trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:boolean-hotspot]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-boolean-hotspot-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-boolean-hotspot-dtest/lastSuccessfulBuild/]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12342) CLibrary improvements

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12342:

Reviewer: Branimir Lambov

> CLibrary improvements
> -
>
> Key: CASSANDRA-12342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12342
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> {{CLibrary}} uses {{FBUtilities.getProtectedField}} for each invocation of 
> {{getfd}} - i.e. {{Class.getDeclaredField}} + {{Field.setAccessible}}. Linked 
> patch migrates these {{Field}} references to static class fields + adds 
> constants for the OS. Also adds a tiny optimization for non-linux OSs in 
> {{trySync}}.
> ||trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:CLibrary-opts]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-CLibrary-opts-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-CLibrary-opts-dtest/lastSuccessfulBuild/]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12271) NonSystemKeyspaces jmx attribute needs to return jre list

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12271:

Reviewer: Chris Lohfink

> NonSystemKeyspaces jmx attribute needs to return jre list
> -
>
> Key: CASSANDRA-12271
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12271
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Lohfink
>Assignee: Edward Ribeiro
>  Labels: lhf
> Attachments: CASSANDRA-12271.patch, screenshot-1.png, screenshot-2.png
>
>
> If you dont have right guava in classpath you cant query the 
> NonSystemKeyspaces attribute. i.e. jconsole. can reproduce using Swiss java 
> knife:
> {code}
> # java -jar sjk.jar mx -s localhost:7199 -mg -b 
> "org.apache.cassandra.db:type=StorageService" -f NonSystemKeyspaces
> org.apache.cassandra.db:type=StorageService
> java.rmi.UnmarshalException: error unmarshalling return; nested exception is: 
>   java.lang.ClassNotFoundException: 
> com.google.common.collect.ImmutableList$SerializedForm (no security manager: 
> RMI class loader disabled)
> {code}
> If return a ArrayList or LinkedList or anything in JRE this will be fixed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12261) dtest failure in write_failures_test.TestWriteFailures.test_thrift

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12261:

Reviewer: Benjamin Lerer

> dtest failure in write_failures_test.TestWriteFailures.test_thrift
> --
>
> Key: CASSANDRA-12261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12261
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_novnode_dtest/14/testReport/write_failures_test/TestWriteFailures/test_thrift
> Failure is
> {code}
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,127 LogTransaction.java:205 - 
> Unable to delete 
> /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-2-big-Data.db
>  as it does not exist
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,334 LogTransaction.java:205 - 
> Unable to delete 
> /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-15-big-Data.db
>  as it does not exist
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,337 LogTransaction.java:205 - 
> Unable to delete 
> /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-31-big-Data.db
>  as it does not exist
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,339 LogTransaction.java:205 - 
> Unable to delete 
> /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-18-big-Data.db
>  as it does not exist
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11889) LogRecord: file system race condition may cause verify() to fail

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11889:

Reviewer: Benjamin Lerer

> LogRecord: file system race condition may cause verify() to fail
> 
>
> Key: CASSANDRA-11889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11889
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> The following exception was reported in CASSANDRA-11470. It occurred whilst 
> listing files with compaction in progress:
> {code}
> WARN  [CompactionExecutor:2006] 2016-05-23 18:23:31,694 
> BigTableWriter.java:171 - Writing large partition 
> test_keyspace/test_columnfamily:eda6b9c36f8df6fe596492c3438d7a38e9b109a6 
> (123663388 bytes)
> INFO  [IndexSummaryManager:1] 2016-05-23 18:24:23,731 
> IndexSummaryRedistribution.java:74 - Redistributing index summaries
> WARN  [CompactionExecutor:2006] 2016-05-23 18:24:56,669 
> BigTableWriter.java:171 - Writing large partition 
> test_keyspace/test_columnfamily:05b6b424194dd19ab7cfbcd53c4979768cd859e9 
> (256286063 bytes)
> WARN  [CompactionExecutor:2006] 2016-05-23 18:26:23,575 
> BigTableWriter.java:171 - Writing large partition 
> test_keyspace/test_columnfamily:04e9fac15552b9ae77c27a6cb8d0fd11fdcc24d7 
> (212445557 bytes)
> INFO  [CompactionExecutor:2005] 2016-05-23 18:29:26,839 
> LeveledManifest.java:437 - Adding high-level (L3) 
> BigTableReader(path='/data/cassandra/data/test_keyspace/test_columnfamily_2-d29dd71045a811e59aff6776bf484396/ma-61041-big-Data.db')
>  to candidates
> WARN  [CompactionExecutor:2006] 2016-05-23 18:30:34,154 
> BigTableWriter.java:171 - Writing large partition 
> test_keyspace/test_columnfamily:edbe6f178503be90911dbf29a55b97a4b095a9ec 
> (183852539 bytes)
> INFO  [CompactionExecutor:2006] 2016-05-23 18:31:21,080 
> LeveledManifest.java:437 - Adding high-level (L3) 
> BigTableReader(path='/data/cassandra/data/test_keyspace/test_columnfamily_2-d29dd71045a811e59aff6776bf484396/ma-61042-big-Data.db')
>  to candidates
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-23 18:31:21,207 
> LogFile.java:173 - Unexpected files detected for sstable [ma-91034-big], 
> record 
> [REMOVE:[/data/cassandra/data/test_keyspace/test_columnfamily-3996ce80b7ac11e48a9b6776bf484396/ma-91034-big,1463992176000,8][457420186]]:
>  last update time [00:00:00] should have been [08:29:36]
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-23 18:31:21,208 
> ScheduledReporter.java:119 - RuntimeException thrown from 
> GraphiteReporter#report. Exception was suppressed.
> java.lang.RuntimeException: Failed to list files in 
> /data/cassandra/data/test_keyspace/test_columnfamily-3996ce80b7ac11e48a9b6776bf484396
>   at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:57)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:691)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories$SSTableLister.listFiles(Directories.java:662)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories$TrueFilesSizeVisitor.(Directories.java:981)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories.getTrueAllocatedSizeIn(Directories.java:893)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.Directories.trueSnapshotsSize(Directories.java:883) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.trueSnapshotsSize(ColumnFamilyStore.java:2332)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:637) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:634) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:692) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
>  ~[metrics-graphite-3.1.0.jar:3.1.0]
>   at 
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) 
> 

[jira] [Updated] (CASSANDRA-11521) Implement streaming for bulk read requests

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11521:

Reviewer: Sylvain Lebresne

> Implement streaming for bulk read requests
> --
>
> Key: CASSANDRA-11521
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11521
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Stefania
>  Labels: client-impacting, protocolv5
> Fix For: 3.x
>
> Attachments: final-patch-jfr-profiles-1.zip
>
>
> Allow clients to stream data from a C* host, bypassing the coordination layer 
> and eliminating the need to query individual pages one by one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12376) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV1Upgrade_AllVersions_EndsAt_indev_2_2_x.parallel_upgrade_test

[Craig Kodman (JIRA)]

Craig Kodman created CASSANDRA-12376:


 Summary: dtest failure in 
upgrade_tests.upgrade_through_versions_test.ProtoV1Upgrade_AllVersions_EndsAt_indev_2_2_x.parallel_upgrade_test
 Key: CASSANDRA-12376
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12376
 Project: Cassandra
  Issue Type: Test
Reporter: Craig Kodman
Assignee: DS Test Eng


example failure:

http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/276/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV1Upgrade_AllVersions_EndsAt_indev_2_2_x/parallel_upgrade_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12375) dtest failure in read_repair_test.TestReadRepair.test_gcable_tombstone_resurrection_on_range_slice_query

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406486#comment-15406486
 ] 

Joel Knighton commented on CASSANDRA-12375:
---

We reverted the ticket this was was introduced for ([CASSANDRA-11427]) in 
[CASSANDRA-12351].

I don't think we plan on reintroducing this, so it probably makes sense to 
remove the test. Can you confirm [~slebresne]?

> dtest failure in 
> read_repair_test.TestReadRepair.test_gcable_tombstone_resurrection_on_range_slice_query
> 
>
> Key: CASSANDRA-12375
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12375
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_gc.log, node2.log, node2_debug.log, 
> node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_jdk8/291/testReport/read_repair_test/TestReadRepair/test_gcable_tombstone_resurrection_on_range_slice_query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11381) Node running with join_ring=false and authentication can not serve requests

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406471#comment-15406471
 ] 

Joel Knighton commented on CASSANDRA-11381:
---

[~michaelsembwever] - The scenario I was referring to was moving 
{{doAuthSetup}} to before joining for the first node in a fresh cluster.

Thanks for keeping this moving. This is near the top of my review queue and I 
should be able to get to it soon.

> Node running with join_ring=false and authentication can not serve requests
> ---
>
> Key: CASSANDRA-11381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11381
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: 11381-2.1.txt, 11381-2.2.txt, 11381-3.0.txt, 
> 11381-trunk.txt, dtest-11381-trunk.txt
>
>
> Starting up a node with {{-Dcassandra.join_ring=false}} in a cluster that has 
> authentication configured, eg PasswordAuthenticator, won't be able to serve 
> requests. This is because {{Auth.setup()}} never gets called during the 
> startup.
> Without {{Auth.setup()}} having been called in {{StorageService}} clients 
> connecting to the node fail with the node throwing
> {noformat}
> java.lang.NullPointerException
> at 
> org.apache.cassandra.auth.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:119)
> at 
> org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1471)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3505)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3489)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at com.thinkaurelius.thrift.Message.invoke(Message.java:314)
> at 
> com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689)
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The exception thrown from the 
> [code|https://github.com/apache/cassandra/blob/cassandra-2.0.16/src/java/org/apache/cassandra/auth/PasswordAuthenticator.java#L119]
> {code}
> ResultMessage.Rows rows = 
> authenticateStatement.execute(QueryState.forInternalCalls(), new 
> QueryOptions(consistencyForUser(username),
>   
>Lists.newArrayList(ByteBufferUtil.bytes(username;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12374) Can't rebuild SASI index

[Alex Petrov (JIRA)]

Alex Petrov created CASSANDRA-12374:
---

 Summary: Can't rebuild SASI index
 Key: CASSANDRA-12374
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
 Project: Cassandra
  Issue Type: Bug
Reporter: Alex Petrov


There's been no real requirement for that so far. 

As [~beobal] has pointed out, it's not a big issue, since that only could be 
needed when index files are lost, data corruption on disk (hardware issue) has 
occurred or there was a bug that'd require an index rebuild.

During {{rebuild_index}} task, indexes are only "marked" as removed with 
{{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}} 
is called. However, since SASI keeps track of SSTables for the index, it's 
going to filter them out with {{.filter((sstable) -> 
!sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.

If I understand the logic correctly, we have to "invalidate" (drop data) right 
before we re-index them. This is also a blocker for [CASSANDRA-11990] since 
without it we can't have an upgrade path.

I have a patch ready in branch, but since it's a bug, it's better to have it 
released earlier and for all branches affected.

cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7190) Add schema to snapshot manifest

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-7190:
-
Fix Version/s: 3.0.x

> Add schema to snapshot manifest
> ---
>
> Key: CASSANDRA-7190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7190
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, lhf
> Fix For: 3.10, 3.0.x
>
>
> followup from CASSANDRA-6326



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7190) Add schema to snapshot manifest

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406416#comment-15406416
 ] 

Alex Petrov commented on CASSANDRA-7190:


Sure thing.
I'll reopen to keep track if nobody minds.

> Add schema to snapshot manifest
> ---
>
> Key: CASSANDRA-7190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7190
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, lhf
> Fix For: 3.10
>
>
> followup from CASSANDRA-6326



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-7190) Add schema to snapshot manifest

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reopened CASSANDRA-7190:


> Add schema to snapshot manifest
> ---
>
> Key: CASSANDRA-7190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7190
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, lhf
> Fix For: 3.10
>
>
> followup from CASSANDRA-6326



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12264) dtest failure in repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12264:

Reviewer: Paulo Motta

> dtest failure in 
> repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test
> 
>
> Key: CASSANDRA-12264
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12264
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Marcus Eriksson
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/15/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test
> {code}
> Error Message
> 'Repaired at: 0' unexpectedly found in 'SSTable: 
> {code}
> Related failure:
> http://cassci.datastax.com/job/trunk_dtest/1315/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12323) dtest failure in compaction_test.TestCompaction_with_DateTieredCompactionStrategy.bloomfilter_size_test

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12323:

Reviewer: Paulo Motta

> dtest failure in 
> compaction_test.TestCompaction_with_DateTieredCompactionStrategy.bloomfilter_size_test
> ---
>
> Key: CASSANDRA-12323
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12323
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Craig Kodman
>Assignee: Marcus Eriksson
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_novnode_dtest/19/testReport/compaction_test/TestCompaction_with_DateTieredCompactionStrategy/bloomfilter_size_test
> 500352 not less than or equal to 15



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7190) Add schema to snapshot manifest

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406338#comment-15406338
 ] 

Aleksey Yeschenko commented on CASSANDRA-7190:
--

[~ifesdjeen] Cannot get a clean cherry-pick. Can you please cook up a 3.0-based 
patch?

> Add schema to snapshot manifest
> ---
>
> Key: CASSANDRA-7190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7190
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, lhf
> Fix For: 3.10
>
>
> followup from CASSANDRA-6326



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12236) RTE from new CDC column breaks in flight queries.

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12236:
--
Fix Version/s: (was: 3.9)

> RTE from new CDC column breaks in flight queries.
> -
>
> Key: CASSANDRA-12236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12236
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeremiah Jordan
>Assignee: Sylvain Lebresne
> Fix For: 3.8
>
> Attachments: 12236.txt
>
>
> This RTE is not harmless. It will cause the internode connection to break 
> which will cause all in flight requests between these nodes to die/timeout.
> {noformat}
> - Due to changes in schema migration handling and the storage format 
> after 3.0, you will
>   see error messages such as:
>  "java.lang.RuntimeException: Unknown column cdc during 
> deserialization"
>   in your system logs on a mixed-version cluster during upgrades. This 
> error message
>   is harmless and due to the 3.8 nodes having cdc added to their schema 
> tables while
>   the <3.8 nodes do not. This message should cease once all nodes are 
> upgraded to 3.8.
>   As always, refrain from schema changes during cluster upgrades.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10692) Don't remove level info when doing upgradesstables

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-10692:
-
Labels: lcs  (was: )

> Don't remove level info when doing upgradesstables
> --
>
> Key: CASSANDRA-10692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10692
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>  Labels: lcs
> Fix For: 2.1.12, 2.2.4
>
>
> Seems we blow away the level info when doing upgradesstables. Introduced in  
> CASSANDRA-8004



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10422) Avoid anticompaction when doing subrange repair

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-10422:
-
Labels: compaction repair  (was: )

> Avoid anticompaction when doing subrange repair
> ---
>
> Key: CASSANDRA-10422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10422
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Ariel Weisberg
>  Labels: compaction, repair
> Fix For: 2.1.12, 2.2.4, 3.0.1, 3.1
>
>
> If we do split the owned range in say 1000 parts, and then do one repair 
> each, we could potentially anticompact every sstable 1000 times (ie, we 
> anticompact the repaired range out 1000 times). We should avoid 
> anticompacting at all in these cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12347) Gossip 2.0 - broadcast tree for data dissemination

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown reassigned CASSANDRA-12347:
---

Assignee: Jason Brown

> Gossip 2.0 - broadcast tree for data dissemination
> --
>
> Key: CASSANDRA-12347
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12347
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jason Brown
>Assignee: Jason Brown
>
> Description: A broadcast tree (spanning tree) allows an originating node to 
> efficiently send out updates to all of the peers in the cluster by 
> constructing a balanced, self-healing tree based upon the view it gets from 
> the peer sampling service (CASSANDRA-12346). 
> I propose we use an algorithm based on the [Thicket 
> paper|http://www.gsd.inesc-id.pt/%7Ejleitao/pdf/srds10-mario.pdf], which 
> describes a dynamic, self-healing broadcast tree. When a given node needs to 
> send out a message, it dynamically builds a tree for each node in the 
> cluster; thus giving us a unique tree for every node in the cluster (a tree 
> rooted at every cluster node). The trees, of course, would be reusable until 
> the cluster configurations changes or failures are detected (by the mechanism 
> described in the paper). Additionally, Thicket includes a mechanism for 
> load-balancing the trees such that nodes spread out the work amongst 
> themselves.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12370) dtest failure in batch_test.TestBatch.acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406220#comment-15406220
 ] 

Joel Knighton commented on CASSANDRA-12370:
---

Agreed - assigning it to me because I'm checking all of these tomorrow after 
the revert.

> dtest failure in 
> batch_test.TestBatch.acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test
> -
>
> Key: CASSANDRA-12370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12370
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest/664/testReport/batch_test/TestBatch/acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12370) dtest failure in batch_test.TestBatch.acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton reassigned CASSANDRA-12370:
-

Assignee: Joel Knighton  (was: DS Test Eng)

> dtest failure in 
> batch_test.TestBatch.acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test
> -
>
> Key: CASSANDRA-12370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12370
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: Joel Knighton
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest/664/testReport/batch_test/TestBatch/acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12371) INSERT JSON - numbers not accepted for smallint and tinyint

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12371:

Since Version: 2.2.0 beta 1  (was: 3.7)
   Labels: lhf  (was: )
 Priority: Minor  (was: Major)
Fix Version/s: 3.x
   3.0.x
   2.2.x

It looks like when these types were added, there were no updates to 
{{JsonTest.java}} to cover them.  As the error message indicates, we're 
explicitly checking {{instanceof Byte}} and {{instanceof Short}}, which is 
probably not the correct thing to do, although it may be possible to get 
jackson to return these types instead of an Integer.

> INSERT JSON - numbers not accepted for smallint and tinyint
> ---
>
> Key: CASSANDRA-12371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12371
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Apache Cassandra 3.7 (provisioned by instaclustr.com, 
> running on AWS)
>Reporter: Paweł Rychlik
>Priority: Minor
>  Labels: lhf
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Contrary to what is written down on 
> http://cassandra.apache.org/doc/latest/cql/json.html#json-encoding-of-cassandra-data-types,
>  numbers are not an accepted format for tinyints and smallints.
> Steps to reproduce on CQLSH:
> > create table default.test(id text PRIMARY KEY, small smallint, tiny 
> > tinyint);
> > INSERT INTO default.test JSON '{"id":"123","small":11}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Error 
> decoding JSON value for small: Expected a short value, but got a Integer: 11"
> > INSERT INTO default.test JSON '{"id":"123","tiny":11}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Error 
> decoding JSON value for tiny: Expected a byte value, but got a Integer: 11"
> The good news is that when you wrap the numeric values into strings - it 
> works like a charm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12370) dtest failure in batch_test.TestBatch.acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406212#comment-15406212
 ] 

Russ Hatch commented on CASSANDRA-12370:


I think this might just be a symptom of test env problems with the latest 
python driver.

> dtest failure in 
> batch_test.TestBatch.acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test
> -
>
> Key: CASSANDRA-12370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12370
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest/664/testReport/batch_test/TestBatch/acknowledged_by_batchlog_not_set_when_batchlog_write_fails_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12008) Make decommission operations resumable

[Kaide Mu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406203#comment-15406203
 ] 

Kaide Mu commented on CASSANDRA-12008:
--

New dtests patch: 
https://github.com/riptano/cassandra-dtest/compare/master...kdmu:cassandra-12008?expand=1
New implementation patch: 
https://github.com/apache/cassandra/compare/trunk...kdmu:trunk-12008?expand=1

bq. Error while decommissioning node is never printed because the 
ExecutionException is being wrapped in a RuntimeException on unbootstrap, so 
perhaps you can modify unbootstrap to throw ExecutionException | 
InterruptedException and catch that on decomission to wrap in RuntimeException.
Now ExecutionException ｜ InterruptedException is handled directly by 
unbootstrap.

bq. When verifying if the retrieved data is correct on 
resumable_decommission_test, you need to stop either node1 or node3 when 
querying the other otherwise the data may be in only one of these nodes (while 
it must be in both nodes, since RF=2 and N=2).
Instead of stopping and starting nodes, I changed stress read with a CL=TWO, in 
this way I guess we can ensure that node1 and node3 is "replying" to the 
request. Also if we do stop and restart node, it seems the restarted node will 
raise an error due to it is looking on node2 log that restarting node is alive 
which such operation is not possible since node2 is decommissioned.

bq. Instead of counting for decommission_error you can add a self.fail("second 
rebuild should fail") after node2.nodetool('decommission') and on the except 
part perhaps check that the following message is being print on logs Error 
while decommissioning node
I guess I'll use insetead  assertRaises which seems more suitable to ensure 
NodetoolError is raised. WDYT [~pauloricardomg] [~yukim]?

> Make decommission operations resumable
> --
>
> Key: CASSANDRA-12008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12008
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Tom van der Woerdt
>Assignee: Kaide Mu
>Priority: Minor
>
> We're dealing with large data sets (multiple terabytes per node) and 
> sometimes we need to add or remove nodes. These operations are very dependent 
> on the entire cluster being up, so while we're joining a new node (which 
> sometimes takes 6 hours or longer) a lot can go wrong and in a lot of cases 
> something does.
> It would be great if the ability to retry streams was implemented.
> Example to illustrate the problem :
> {code}
> 03:18 PM   ~ $ nodetool decommission
> error: Stream failed
> -- StackTrace --
> org.apache.cassandra.streaming.StreamException: Stream failed
> at 
> org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
> at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
> at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
> at 
> com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
> at 
> com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
> at 
> com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
> at 
> org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:210)
> at 
> org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:186)
> at 
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:430)
> at 
> org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:622)
> at 
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:486)
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:274)
> at java.lang.Thread.run(Thread.java:745)
> 08:04 PM   ~ $ nodetool decommission
> nodetool: Unsupported operation: Node in LEAVING state; wait for status to 
> become normal or restart
> See 'nodetool help' or 'nodetool help '.
> {code}
> Streaming failed, probably due to load :
> {code}
> ERROR [STREAM-IN-/] 2016-06-14 18:05:47,275 StreamSession.java:520 - 
> [Stream #] Streaming error occurred
> java.net.SocketTimeoutException: null
> at 
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:211) 
> ~[na:1.8.0_77]
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) 
> ~[na:1.8.0_77]
> at 
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385) 
> ~[na:1.8.0_77]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> 

[jira] [Commented] (CASSANDRA-12312) Restore JVM metric export for metric reporters

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406188#comment-15406188
 ] 

T Jake Luciani commented on CASSANDRA-12312:


bq. what do you think about including metrics on thread states based on 
ThreadStatesGaugeSet.java?

I think this might be overkill since we can use jstack for the same.  It 
doesn't seem valuable to see all this info all the time.

> Restore JVM metric export for metric reporters
> --
>
> Key: CASSANDRA-12312
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12312
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>  Labels: lhf
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 12312-2.2.patch, 12312-3.0.patch, 12312-trunk.patch, 
> metrics-jvm-3.1.0.jar.asc
>
>
> JVM instrumentation as part of dropwizard metrics has been moved to a 
> separate {{metrics-jvm}} artifact in metrics-v3.0. After CASSANDRA-5657, no 
> jvm related metrics will be exported to any reporter configured via 
> {{metrics-reporter-config}}, as this isn't part of {{metrics-core}} anymore. 
> As memory and GC stats are essential for monitoring Cassandra, this turns out 
> to be a blocker for us for upgrading to 2.2.
> I've included a patch that would add the now separate {{metrics-jvm}} package 
> and enables some of the provided metrics on startup in case a metrics 
> reporter is used ({{-Dcassandra.metricsReporterConfigFile}}).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12312) Restore JVM metric export for metric reporters

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406184#comment-15406184
 ] 

T Jake Luciani commented on CASSANDRA-12312:


Great thanks,  I'll commit this soon.

> Restore JVM metric export for metric reporters
> --
>
> Key: CASSANDRA-12312
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12312
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>  Labels: lhf
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 12312-2.2.patch, 12312-3.0.patch, 12312-trunk.patch, 
> metrics-jvm-3.1.0.jar.asc
>
>
> JVM instrumentation as part of dropwizard metrics has been moved to a 
> separate {{metrics-jvm}} artifact in metrics-v3.0. After CASSANDRA-5657, no 
> jvm related metrics will be exported to any reporter configured via 
> {{metrics-reporter-config}}, as this isn't part of {{metrics-core}} anymore. 
> As memory and GC stats are essential for monitoring Cassandra, this turns out 
> to be a blocker for us for upgrading to 2.2.
> I've included a patch that would add the now separate {{metrics-jvm}} package 
> and enables some of the provided metrics on startup in case a metrics 
> reporter is used ({{-Dcassandra.metricsReporterConfigFile}}).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12092) dtest failure in consistency_test.TestAccuracy.test_simple_strategy_counters

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch updated CASSANDRA-12092:
---
Assignee: DS Test Eng  (was: Russ Hatch)

> dtest failure in consistency_test.TestAccuracy.test_simple_strategy_counters
> 
>
> Key: CASSANDRA-12092
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12092
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/484/testReport/consistency_test/TestAccuracy/test_simple_strategy_counters
> Failed on CassCI build cassandra-2.1_dtest #484
> {code}
> Standard Error
> Traceback (most recent call last):
>   File "/home/automaton/cassandra-dtest/consistency_test.py", line 514, in run
> valid_fcn(v)
>   File "/home/automaton/cassandra-dtest/consistency_test.py", line 497, in 
> validate_counters
> check_all_sessions(s, n, c)
>   File "/home/automaton/cassandra-dtest/consistency_test.py", line 490, in 
> check_all_sessions
> "value of %s at key %d, instead got these values: %s" % (write_nodes, 
> val, n, results)
> AssertionError: Failed to read value from sufficient number of nodes, 
> required 2 nodes to have a counter value of 1 at key 200, instead got these 
> values: [0, 0, 1]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12312) Restore JVM metric export for metric reporters

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-12312:
---
Status: Ready to Commit  (was: Patch Available)

> Restore JVM metric export for metric reporters
> --
>
> Key: CASSANDRA-12312
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12312
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>  Labels: lhf
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 12312-2.2.patch, 12312-3.0.patch, 12312-trunk.patch, 
> metrics-jvm-3.1.0.jar.asc
>
>
> JVM instrumentation as part of dropwizard metrics has been moved to a 
> separate {{metrics-jvm}} artifact in metrics-v3.0. After CASSANDRA-5657, no 
> jvm related metrics will be exported to any reporter configured via 
> {{metrics-reporter-config}}, as this isn't part of {{metrics-core}} anymore. 
> As memory and GC stats are essential for monitoring Cassandra, this turns out 
> to be a blocker for us for upgrading to 2.2.
> I've included a patch that would add the now separate {{metrics-jvm}} package 
> and enables some of the provided metrics on startup in case a metrics 
> reporter is used ({{-Dcassandra.metricsReporterConfigFile}}).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12366) Fix compaction throttle

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-12366:
---
Status: Patch Available  (was: Open)

> Fix compaction throttle
> ---
>
> Key: CASSANDRA-12366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12366
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.x
>
>
> Compaction throttling is broken in the following ways:
>   * It throttles bytes read after being decompressed
>   * Compaction creates multiple scanners which share the rate limiter causing 
> too much throttling
>   * It bears no resemblance to the reported compaction time remaining 
> calculation (Bytes of source sstables processed since start of compaction)
> To fix this we need to simplify the throttling to be only at the 
> CompactionIterator level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12366) Fix compaction throttle

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406166#comment-15406166
 ] 

T Jake Luciani commented on CASSANDRA-12366:


I pushed an update to fix the compression ratio calc. Also, I noticed the way 
of calculating the size read was broken for leveled compactions since we 
combine many sstables into the scanner.  When a reader jumps from one sstable 
to another there may be big jumps which don't reflect the actual read amount.

I also added this to cleanup, verify, gc compactions. 

> Fix compaction throttle
> ---
>
> Key: CASSANDRA-12366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12366
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.x
>
>
> Compaction throttling is broken in the following ways:
>   * It throttles bytes read after being decompressed
>   * Compaction creates multiple scanners which share the rate limiter causing 
> too much throttling
>   * It bears no resemblance to the reported compaction time remaining 
> calculation (Bytes of source sstables processed since start of compaction)
> To fix this we need to simplify the throttling to be only at the 
> CompactionIterator level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8988) Optimise IntervalTree

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-8988:

Labels: compaction performance  (was: performance)

> Optimise IntervalTree
> -
>
> Key: CASSANDRA-8988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8988
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
>Priority: Minor
>  Labels: compaction, performance
> Fix For: 2.2.0 beta 1
>
> Attachments: 8988.txt
>
>
> We perform a lot of unnecessary comparisons in 
> IntervalTree.IntervalNode.searchInternal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12347) Gossip 2.0 - broadcast tree for data dissemination

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406163#comment-15406163
 ] 

Tyler Hobbs commented on CASSANDRA-12347:
-

It's been a while since we discussed this, so I apologize if my understanding 
of things is rusty.  I definitely see the value in CASSANDRA-12346, and think 
it's an excellent step forward.  However, I'm not as convinced yet about this 
ticket.  Is it fair to say that these are the two main reasons for using 
broadcast trees?
# Minimize the amount of time it takes for changes to be disseminated 
throughout the cluster
# Minimize the amount of redundant messaging, thereby minimizing the amount of 
gossip traffic

In the past, we discussed addressing #2 by reserving Gossip for infrequent 
changes (schema, topology) and basing the failure detector on coordinated 
requests instead of gossip heartbeats. That, combined with removing "severity" 
from Gossip (which we recently started ignoring anyway) would make Gossip 
traffic quite low already, leaving #2 as a pretty minor concern.

Regarding #1, assuming that we only use Gossip for relatively infrequent 
changes, shouldn't broadcasting to every node in the "active view" (in 
HyParView terms, CASSANDRA-12346) achieve rapid dissemination on its own?

Thicket is considerably more complex than just HyParView, so I want to make 
sure that we're getting good enough benefits out of the additional complexity.

> Gossip 2.0 - broadcast tree for data dissemination
> --
>
> Key: CASSANDRA-12347
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12347
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jason Brown
>
> Description: A broadcast tree (spanning tree) allows an originating node to 
> efficiently send out updates to all of the peers in the cluster by 
> constructing a balanced, self-healing tree based upon the view it gets from 
> the peer sampling service (CASSANDRA-12346). 
> I propose we use an algorithm based on the [Thicket 
> paper|http://www.gsd.inesc-id.pt/%7Ejleitao/pdf/srds10-mario.pdf], which 
> describes a dynamic, self-healing broadcast tree. When a given node needs to 
> send out a message, it dynamically builds a tree for each node in the 
> cluster; thus giving us a unique tree for every node in the cluster (a tree 
> rooted at every cluster node). The trees, of course, would be reusable until 
> the cluster configurations changes or failures are detected (by the mechanism 
> described in the paper). Additionally, Thicket includes a mechanism for 
> load-balancing the trees such that nodes spread out the work amongst 
> themselves.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8920) Optimise sequential overlap visitation for checking tombstone retention in compaction

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-8920:

Labels: compaction lcs performance  (was: performance)

> Optimise sequential overlap visitation for checking tombstone retention in 
> compaction
> -
>
> Key: CASSANDRA-8920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8920
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
>Priority: Minor
>  Labels: compaction, lcs, performance
> Fix For: 2.2.0 beta 1
>
> Attachments: 8920.txt
>
>
> The IntervalTree only maps partition keys. Since a majority of users deploy a 
> hashed partitioner the work is mostly wasted, since they will be evenly 
> distributed across the full token range owned by the node - and in some cases 
> it is a significant amount of work. We can perform a corroboration against 
> the file bounds if we get a BF match as a sanity check if we like, but 
> performing an IntervalTree search is significantly more expensive (esp. once 
> murmur hash calculation memoization goes mainstream).
> In LCS, the keys are bounded, to it might appear that it would help, but in 
> this scenario we only compact against like bounds, so again it is not helpful.
> With a ByteOrderedPartitioner it could potentially be of use, but this is 
> sufficiently rare to not optimise for IMO.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8915) Improve MergeIterator performance

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-8915:

Labels: compaction lcs performance  (was: compaction performance)

> Improve MergeIterator performance
> -
>
> Key: CASSANDRA-8915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8915
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Minor
>  Labels: compaction, lcs, performance
> Fix For: 3.0 alpha 1
>
>  Time Spent: 0.2h
>  Remaining Estimate: 0h
>
> The implementation of {{MergeIterator}} uses a priority queue and applies a 
> pair of {{poll}}+{{add}} operations for every item in the resulting sequence. 
> This is quite inefficient as {{poll}} necessarily applies at least {{log N}} 
> comparisons (up to {{2log N}}), and {{add}} often requires another {{log N}}, 
> for example in the case where the inputs largely don't overlap (where {{N}} 
> is the number of iterators being merged).
> This can easily be replaced with a simple custom structure that can perform 
> replacement of the top of the queue in a single step, which will very often 
> complete after a couple of comparisons and in the worst case scenarios will 
> match the complexity of the current implementation.
> This should significantly improve merge performance for iterators with 
> limited overlap (e.g. levelled compaction).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10993) Make read and write requests paths fully non-blocking, eliminate related stages

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406130#comment-15406130
 ] 

Sylvain Lebresne commented on CASSANDRA-10993:
--

I won't pretend having read very carefully neither this patch nor the one for 
RxJava but the argument that with little proof of performance advantage we're 
better off leveraging and existing and tested library, especially when the API 
is getting somewhat standardized, resonates with me. I also share the gut 
feeling that the bulk of the wins is going to actually get to TPC (or as close 
to it as we can) so we can simplify data structures (not only in term of 
performance, but also in term of complexity required by these data structures), 
and re-using an existing library sounds like it can get us there sooner without 
too much drawback.

> Make read and write requests paths fully non-blocking, eliminate related 
> stages
> ---
>
> Key: CASSANDRA-10993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10993
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Tyler Hobbs
> Fix For: 3.x
>
> Attachments: 10993-reads-no-evloop-integration-six-node-stress.svg, 
> tpc-benchmarks-2.txt, tpc-benchmarks.txt
>
>
> Building on work done by [~tjake] (CASSANDRA-10528), [~slebresne] 
> (CASSANDRA-5239), and others, convert read and write request paths to be 
> fully non-blocking, to enable the eventual transition from SEDA to TPC 
> (CASSANDRA-10989)
> Eliminate {{MUTATION}}, {{COUNTER_MUTATION}}, {{VIEW_MUTATION}}, {{READ}}, 
> and {{READ_REPAIR}} stages, move read and write execution directly to Netty 
> context.
> For lack of decent async I/O options on Linux, we’ll still have to retain an 
> extra thread pool for serving read requests for data not residing in our page 
> cache (CASSANDRA-5863), however.
> Implementation-wise, we only have two options available to us: explicit FSMs 
> and chained futures. Fibers would be the third, and easiest option, but 
> aren’t feasible in Java without resorting to direct bytecode manipulation 
> (ourselves or using [quasar|https://github.com/puniverse/quasar]).
> I have seen 4 implementations bases on chained futures/promises now - three 
> in Java and one in C++ - and I’m not convinced that it’s the optimal (or 
> sane) choice for representing our complex logic - think 2i quorum read 
> requests with timeouts at all levels, read repair (blocking and 
> non-blocking), and speculative retries in the mix, {{SERIAL}} reads and 
> writes.
> I’m currently leaning towards an implementation based on explicit FSMs, and 
> intend to provide a prototype - soonish - for comparison with 
> {{CompletableFuture}}-like variants.
> Either way the transition is a relatively boring straightforward refactoring.
> There are, however, some extension points on both write and read paths that 
> we do not control:
> - authorisation implementations will have to be non-blocking. We have control 
> over built-in ones, but for any custom implementation we will have to execute 
> them in a separate thread pool
> - 2i hooks on the write path will need to be non-blocking
> - any trigger implementations will not be allowed to block
> - UDFs and UDAs
> We are further limited by API compatibility restrictions in the 3.x line, 
> forbidding us to alter, or add any non-{{default}} interface methods to those 
> extension points, so these pose a problem.
> Depending on logistics, expecting to get this done in time for 3.4 or 3.6 
> feature release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12363) dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.list_test

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch resolved CASSANDRA-12363.

Resolution: Cannot Reproduce

100 iterations with no repro. Resolving.

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.list_test
> --
>
> Key: CASSANDRA-12363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12363
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/5/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/list_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line 
> 1375, in list_test
> assert_one(cursor, "SELECT tags FROM user", [['foo', 'bar', 'foo', 
> 'foobar']])
>   File "/home/automaton/cassandra-dtest/assertions.py", line 124, in 
> assert_one
> assert list_res == [expected], "Expected {} from {}, but got 
> {}".format([expected], query, list_res)
> "Expected [[['foo', 'bar', 'foo', 'foobar']]] from SELECT tags FROM user, but 
> got [[[u'foo', u'foo', u'bar', u'foo', u'foobar']]]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11887) Duplicate rows after a 2.2.5 to 3.0.4 migration

[Jesse Hodges (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406107#comment-15406107
 ] 

Jesse Hodges commented on CASSANDRA-11887:
--

Noticed today that if i include the final clustering column in the query, the 
results look correct. 

{code}
cqlsh:alarms> select * from alarms.last_seen_state  where partition_id=10 and 
alarm_id='59893' and tenant_id = f50f8413-57bb-4eb5-a37c-7482a63ea9a5 and 
account_id = '10303' and source='PORTAL';

 partition_id | alarm_id | tenant_id| account_id | 
source | metric | last_seen   | value
--+--+--++++-+--
   10 |59893 | f50f8413-57bb-4eb5-a37c-7482a63ea9a5 |  10303 | 
PORTAL |CPU | 2016-08-03 14:57:48.00+ | 0.939592
   10 |59893 | f50f8413-57bb-4eb5-a37c-7482a63ea9a5 |  10303 | 
PORTAL |CPU | 2016-08-01 15:27:37.00+ |1
   10 |59893 | f50f8413-57bb-4eb5-a37c-7482a63ea9a5 |  10303 | 
PORTAL |CPU | 2016-08-01 15:07:15.00+ |1

(3 rows)
cqlsh:alarms> select * from alarms.last_seen_state  where partition_id=10 and 
alarm_id='59893' and tenant_id = f50f8413-57bb-4eb5-a37c-7482a63ea9a5 and 
account_id = '10303' and source='PORTAL' AND METRIC='CPU';

 partition_id | alarm_id | tenant_id| account_id | 
source | metric | last_seen   | value
--+--+--++++-+--
   10 |59893 | f50f8413-57bb-4eb5-a37c-7482a63ea9a5 |  10303 | 
PORTAL |CPU | 2016-08-03 14:57:48.00+ | 0.939592
{code}

> Duplicate rows after a 2.2.5 to 3.0.4 migration
> ---
>
> Key: CASSANDRA-11887
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11887
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Julien Anguenot
>Priority: Blocker
> Fix For: 3.0.x
>
>
> After migrating from 2.2.5 to 3.0.4, some tables seem to carry duplicate 
> primary keys.
> Below an example. Note, repair / scrub of such table do not seem to fix nor 
> indicate any issues.
> *Table definition*:
> {code}
> CREATE TABLE core.edge_ipsec_vpn_service (
> edge_uuid text PRIMARY KEY,
> enabled boolean,
> endpoints set,
> tunnels set
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> *UDTs:*
> {code}
> CREATE TYPE core.edge_ipsec_vpn_endpoint (
> network text,
> public_ip text
> );
> CREATE TYPE core.edge_ipsec_vpn_tunnel (
> name text,
> description text,
> peer_ip_address text,
> peer_id text,
> local_ip_address text,
> local_id text,
> local_subnets frozen>,
> peer_subnets frozen>,
> shared_secret text,
> shared_secret_encrypted boolean,
> encryption_protocol text,
> mtu int,
> enabled boolean,
> operational boolean,
> error_details text,
> vpn_peer frozen
> );
> CREATE TYPE core.edge_ipsec_vpn_subnet (
> name text,
> gateway text,
> netmask text
> );
> CREATE TYPE core.edge_ipsec_vpn_peer (
> type text,
> id text,
> name text,
> vcd_url text,
> vcd_org text,
> vcd_username text
> );
> {code}
> sstabledump extract (IP addressees hidden as well as  secrets)
> {code}
> [...]
>  {
> "partition" : {
>   "key" : [ "84d567cc-0165-4e64-ab97-3a9d06370ba9" ],
>   "position" : 131146
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 131236,
> "liveness_info" : { "tstamp" : "2016-05-06T17:07:15.416003Z" },
> "cells" : [
>   { "name" : "enabled", "value" : "true" },
>   { "name" : "tunnels", "path" : [ 
> “XXX::1.2.3.4:1.2.3.4:1.2.3.4:1.2.3.4:XXX:XXX:false:AES256:1500:true:false::third
>  party\\:1.2.3.4\\:\\:\\:\\:” ], "value" : "" }
> ]
>   },
>   {
> "type" : "row",
> "position" : 131597,
> 

[jira] [Updated] (CASSANDRA-12236) RTE from new CDC column breaks in flight queries.

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-12236:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Alright, so as far as this issue is concerned, the unit and dtest runs were 
clean (outside of the dtest failure on cqlsh DESCRIBED due to the cdc option 
not being there by default and for which I've committed a fix to the dtest 
for), so committed, thanks.

It's a bit unclear how much upgrade tests this actually fix as there is a bit 
too much noise currently, but it at the very least remove one know reason for 
failure.

> RTE from new CDC column breaks in flight queries.
> -
>
> Key: CASSANDRA-12236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12236
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeremiah Jordan
>Assignee: Sylvain Lebresne
> Fix For: 3.8, 3.9
>
> Attachments: 12236.txt
>
>
> This RTE is not harmless. It will cause the internode connection to break 
> which will cause all in flight requests between these nodes to die/timeout.
> {noformat}
> - Due to changes in schema migration handling and the storage format 
> after 3.0, you will
>   see error messages such as:
>  "java.lang.RuntimeException: Unknown column cdc during 
> deserialization"
>   in your system logs on a mixed-version cluster during upgrades. This 
> error message
>   is harmless and due to the 3.8 nodes having cdc added to their schema 
> tables while
>   the <3.8 nodes do not. This message should cease once all nodes are 
> upgraded to 3.8.
>   As always, refrain from schema changes during cluster upgrades.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[03/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java 
b/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
index 2de671c..72441cd 100644
--- a/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
+++ b/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.cql3.ColumnIdentifier;
 import org.apache.cassandra.cql3.FieldIdentifier;
 import org.apache.cassandra.cql3.functions.*;
 import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.index.TargetParser;
 import org.apache.cassandra.thrift.ThriftConversion;
@@ -564,35 +565,40 @@ public class LegacySchemaMigratorTest
 setLegacyIndexStatus(keyspace);
 }
 
-private static Mutation makeLegacyCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+private static DecoratedKey decorate(CFMetaData metadata, Object value)
 {
-// Note that because Keyspaces is a COMPACT TABLE, we're really only 
setting static columns internally and shouldn't set any clustering.
-RowUpdateBuilder adder = new 
RowUpdateBuilder(SystemKeyspace.LegacyKeyspaces, timestamp, keyspace.name);
+return 
metadata.decorateKey(((AbstractType)metadata.getKeyValidator()).decompose(value));
+}
 
-adder.add("durable_writes", keyspace.params.durableWrites)
- .add("strategy_class", 
keyspace.params.replication.klass.getName())
- .add("strategy_options", 
json(keyspace.params.replication.options));
+private static Mutation makeLegacyCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+{
+Mutation.SimpleBuilder builder = 
Mutation.simpleBuilder(SystemKeyspace.NAME, 
decorate(SystemKeyspace.LegacyKeyspaces, keyspace.name))
+ .timestamp(timestamp);
 
-Mutation mutation = adder.build();
+builder.update(SystemKeyspace.LegacyKeyspaces)
+   .row()
+   .add("durable_writes", keyspace.params.durableWrites)
+   .add("strategy_class", 
keyspace.params.replication.klass.getName())
+   .add("strategy_options", 
json(keyspace.params.replication.options));
 
-keyspace.tables.forEach(table -> addTableToSchemaMutation(table, 
timestamp, true, mutation));
-keyspace.types.forEach(type -> addTypeToSchemaMutation(type, 
timestamp, mutation));
-keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, timestamp, mutation));
-keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, timestamp, mutation));
+keyspace.tables.forEach(table -> addTableToSchemaMutation(table, true, 
builder));
+keyspace.types.forEach(type -> addTypeToSchemaMutation(type, builder));
+keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, builder));
+keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, builder));
 
-return mutation;
+return builder.build();
 }
 
 /*
  * Serializing tables
  */
 
-private static void addTableToSchemaMutation(CFMetaData table, long 
timestamp, boolean withColumnsAndTriggers, Mutation mutation)
+private static void addTableToSchemaMutation(CFMetaData table, boolean 
withColumnsAndTriggers, Mutation.SimpleBuilder builder)
 {
 // For property that can be null (and can be changed), we insert 
tombstones, to make sure
 // we don't keep a property the user has removed
-RowUpdateBuilder adder = new 
RowUpdateBuilder(SystemKeyspace.LegacyColumnfamilies, timestamp, mutation)
- .clustering(table.cfName);
+Row.SimpleBuilder adder = 
builder.update(SystemKeyspace.LegacyColumnfamilies)
+ .row(table.cfName);
 
 adder.add("cf_id", table.cfId)
  .add("type", table.isSuper() ? "Super" : "Standard");
@@ -625,12 +631,14 @@ public class LegacySchemaMigratorTest
  .add("read_repair_chance", table.params.readRepairChance)
  .add("speculative_retry", 
table.params.speculativeRetry.toString());
 
+Map dropped = new HashMap<>();
 for (Map.Entry entry : 
table.getDroppedColumns().entrySet())
 {
 String name = UTF8Type.instance.getString(entry.getKey());
 CFMetaData.DroppedColumn column = entry.getValue();
-adder.addMapEntry("dropped_columns", name, column.droppedTime);
+dropped.put(name, column.droppedTime);
 }
+adder.add("dropped_columns", 

[08/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

Fix RTE on mixed-version cluster due to CDC schema changes.

Patch by jmckenzie and slebresne; reviewed by ayeschenko for CASSANDRA-12236


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/26838063
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/26838063
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/26838063

Branch: refs/heads/cassandra-3.8
Commit: 26838063de6246e3a1e18062114ca92fb81c00cf
Parents: b27e2f9
Author: Josh McKenzie 
Authored: Thu Jul 21 12:45:13 2016 -0400
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:41:24 2016 +0200

--
 CHANGES.txt |   1 +
 NEWS.txt|  10 +-
 .../cassandra/batchlog/BatchlogManager.java |  19 +-
 .../batchlog/LegacyBatchlogMigrator.java|   9 +-
 src/java/org/apache/cassandra/db/Mutation.java  |  66 +++
 .../apache/cassandra/db/RowUpdateBuilder.java   | 400 
 .../org/apache/cassandra/db/SimpleBuilders.java | 461 +++
 .../org/apache/cassandra/db/SystemKeyspace.java |  11 +-
 .../db/partitions/AbstractBTreePartition.java   |   2 +-
 .../db/partitions/PartitionUpdate.java  | 154 +++
 src/java/org/apache/cassandra/db/rows/Row.java  |  99 
 src/java/org/apache/cassandra/db/rows/Rows.java |  16 +
 .../apache/cassandra/db/transform/BaseRows.java |   3 +-
 .../cassandra/schema/LegacySchemaMigrator.java  |  12 +-
 .../apache/cassandra/schema/SchemaKeyspace.java | 427 +
 .../cassandra/service/MigrationManager.java |   8 +-
 .../apache/cassandra/tracing/TraceKeyspace.java |  52 ++-
 .../org/apache/cassandra/UpdateBuilder.java |  56 +--
 test/unit/org/apache/cassandra/Util.java|  26 +-
 .../apache/cassandra/batchlog/BatchTest.java|  17 +-
 .../apache/cassandra/config/CFMetaDataTest.java |   2 +-
 .../apache/cassandra/cql3/CDCStatementTest.java |  10 +
 .../entities/RowUpdateBuilderTest.java  |  79 
 .../db/RecoveryManagerMissingHeaderTest.java|   4 +-
 .../cassandra/db/RecoveryManagerTest.java   |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   | 196 
 .../cassandra/db/compaction/TTLExpiryTest.java  |   2 +-
 .../db/partition/PartitionUpdateTest.java   |  23 +-
 .../org/apache/cassandra/hints/HintTest.java|  56 +--
 .../hints/LegacyHintsMigratorTest.java  |   3 +-
 .../org/apache/cassandra/schema/DefsTest.java   |   2 +-
 .../schema/LegacySchemaMigratorTest.java| 111 ++---
 .../cassandra/schema/SchemaKeyspaceTest.java|   6 +-
 .../cassandra/service/DataResolverTest.java |   2 +-
 .../streaming/StreamingTransferTest.java|   2 +-
 35 files changed, 1448 insertions(+), 907 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4330fde..388a290 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.8
+ * RTE from new CDC column breaks in flight queries (CASSANDRA-12236)
  * Fix hdr logging for single operation workloads (CASSANDRA-12145)
  * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 7418f3a..d8d84f5 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -39,6 +39,9 @@ New features
  the data/cdc_raw directory until removed by the user and writes to 
CDC-enabled tables
  will be rejected with a WriteTimeoutException once cdc_total_space_in_mb 
is reached
  between unflushed CommitLogSegments and cdc_raw.
+ NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on 
a mixed-version
+ cluster as it will lead to exceptions which can interrupt traffic. Once 
all nodes
+ have been upgraded to 3.8 it is safe to enable this feature and restart 
the cluster.
 
 Upgrading
 -
@@ -48,13 +51,6 @@ Upgrading
   those under a different name, change your code to use the new names and
   drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 
for more
   details).
-- Due to changes in schema migration handling and the storage format after 
3.0, you will
-  see error messages such as:
- "java.lang.RuntimeException: Unknown column cdc during 
deserialization"
-  in your system logs on a mixed-version cluster during upgrades. This 
error message
-  is harmless and due to the 3.8 nodes having cdc added to their schema 
tables while
-  the <3.8 nodes do not. This 

[10/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index 8e3961e..7a90dab 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -266,8 +266,8 @@ public final class SchemaKeyspace
 }
 
 // (+1 to timestamp to make sure we don't get shadowed by the 
tombstones we just added)
-makeCreateKeyspaceMutation(system, timestamp + 1).apply();
-makeCreateKeyspaceMutation(schema, timestamp + 1).apply();
+makeCreateKeyspaceMutation(system, timestamp + 1).build().apply();
+makeCreateKeyspaceMutation(schema, timestamp + 1).build().apply();
 }
 
 public static void truncate()
@@ -377,11 +377,6 @@ public final class SchemaKeyspace
 }
 }
 
-private static ByteBuffer getSchemaKSKey(String ksName)
-{
-return AsciiType.instance.fromString(ksName);
-}
-
 private static boolean isSystemKeyspaceSchemaPartition(DecoratedKey 
partitionKey)
 {
 return 
Schema.isSystemKeyspace(UTF8Type.instance.compose(partitionKey.getKey()));
@@ -391,152 +386,152 @@ public final class SchemaKeyspace
  * Schema entities to mutations
  */
 
-public static Mutation makeCreateKeyspaceMutation(String name, 
KeyspaceParams params, long timestamp)
+private static DecoratedKey decorate(CFMetaData metadata, Object value)
 {
-RowUpdateBuilder adder = new RowUpdateBuilder(Keyspaces, timestamp, 
name).clustering();
-return adder.add(KeyspaceParams.Option.DURABLE_WRITES.toString(), 
params.durableWrites)
-.frozenMap(KeyspaceParams.Option.REPLICATION.toString(), 
params.replication.asMap())
-.build();
+return 
metadata.decorateKey(((AbstractType)metadata.getKeyValidator()).decompose(value));
 }
 
-public static Mutation makeCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+public static Mutation.SimpleBuilder makeCreateKeyspaceMutation(String 
name, KeyspaceParams params, long timestamp)
 {
-Mutation mutation = makeCreateKeyspaceMutation(keyspace.name, 
keyspace.params, timestamp);
+Mutation.SimpleBuilder builder = 
Mutation.simpleBuilder(Keyspaces.ksName, decorate(Keyspaces, name))
+ .timestamp(timestamp);
 
-keyspace.tables.forEach(table -> addTableToSchemaMutation(table, 
timestamp, true, mutation));
-keyspace.views.forEach(view -> addViewToSchemaMutation(view, 
timestamp, true, mutation));
-keyspace.types.forEach(type -> addTypeToSchemaMutation(type, 
timestamp, mutation));
-keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, timestamp, mutation));
-keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, timestamp, mutation));
+builder.update(Keyspaces)
+   .row()
+   .add(KeyspaceParams.Option.DURABLE_WRITES.toString(), 
params.durableWrites)
+   .add(KeyspaceParams.Option.REPLICATION.toString(), 
params.replication.asMap());
 
-return mutation;
+return builder;
 }
 
-public static Mutation makeDropKeyspaceMutation(KeyspaceMetadata keyspace, 
long timestamp)
+public static Mutation.SimpleBuilder 
makeCreateKeyspaceMutation(KeyspaceMetadata keyspace, long timestamp)
 {
-int nowInSec = FBUtilities.nowInSeconds();
-Mutation mutation = new Mutation(NAME, 
Keyspaces.decorateKey(getSchemaKSKey(keyspace.name)));
+Mutation.SimpleBuilder builder = 
makeCreateKeyspaceMutation(keyspace.name, keyspace.params, timestamp);
 
-for (CFMetaData schemaTable : ALL_TABLE_METADATA)
-mutation.add(PartitionUpdate.fullPartitionDelete(schemaTable, 
mutation.key(), timestamp, nowInSec));
+keyspace.tables.forEach(table -> addTableToSchemaMutation(table, true, 
builder));
+keyspace.views.forEach(view -> addViewToSchemaMutation(view, true, 
builder));
+keyspace.types.forEach(type -> addTypeToSchemaMutation(type, builder));
+keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, builder));
+keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, builder));
 
-return mutation;
+return builder;
 }
 
-public static Mutation makeCreateTypeMutation(KeyspaceMetadata keyspace, 
UserType type, long timestamp)
+public static Mutation.SimpleBuilder 
makeDropKeyspaceMutation(KeyspaceMetadata keyspace, long timestamp)
 {
-// Include the serialized keyspace in case the target node missed a 
CREATE KEYSPACE migration (see CASSANDRA-5631).
-Mutation 

[14/14] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[slebresne]

Merge branch 'cassandra-3.9' into trunk

* cassandra-3.9:
  Fix RTE on mixed-version cluster due to CDC schema changes.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4e21f726
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4e21f726
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4e21f726

Branch: refs/heads/trunk
Commit: 4e21f7267dc051fdd8214c67859e639ee5a9824f
Parents: 5cc0d15 b603720
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:44:44 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:44:44 2016 +0200

--
 CHANGES.txt |   5 +
 NEWS.txt|  10 +-
 .../cassandra/batchlog/BatchlogManager.java |  19 +-
 .../batchlog/LegacyBatchlogMigrator.java|   9 +-
 src/java/org/apache/cassandra/db/Mutation.java  |  66 +++
 .../apache/cassandra/db/RowUpdateBuilder.java   | 400 
 .../org/apache/cassandra/db/SimpleBuilders.java | 461 +++
 .../org/apache/cassandra/db/SystemKeyspace.java |  11 +-
 .../db/partitions/AbstractBTreePartition.java   |   2 +-
 .../db/partitions/PartitionUpdate.java  | 154 +++
 src/java/org/apache/cassandra/db/rows/Row.java  |  99 
 src/java/org/apache/cassandra/db/rows/Rows.java |  16 +
 .../apache/cassandra/db/transform/BaseRows.java |   3 +-
 .../cassandra/schema/LegacySchemaMigrator.java  |  12 +-
 .../apache/cassandra/schema/SchemaKeyspace.java | 427 +
 .../cassandra/service/MigrationManager.java |   8 +-
 .../apache/cassandra/tracing/TraceKeyspace.java |  52 ++-
 .../org/apache/cassandra/UpdateBuilder.java |  56 +--
 test/unit/org/apache/cassandra/Util.java|  26 +-
 .../apache/cassandra/batchlog/BatchTest.java|  17 +-
 .../apache/cassandra/config/CFMetaDataTest.java |   2 +-
 .../apache/cassandra/cql3/CDCStatementTest.java |  10 +
 .../entities/RowUpdateBuilderTest.java  |  79 
 .../db/RecoveryManagerMissingHeaderTest.java|   4 +-
 .../cassandra/db/RecoveryManagerTest.java   |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   | 196 
 .../cassandra/db/compaction/TTLExpiryTest.java  |   2 +-
 .../db/partition/PartitionUpdateTest.java   |  23 +-
 .../org/apache/cassandra/hints/HintTest.java|  56 +--
 .../hints/LegacyHintsMigratorTest.java  |   3 +-
 .../org/apache/cassandra/schema/DefsTest.java   |   2 +-
 .../schema/LegacySchemaMigratorTest.java| 111 ++---
 .../cassandra/schema/SchemaKeyspaceTest.java|   6 +-
 .../cassandra/service/DataResolverTest.java |   2 +-
 .../streaming/StreamingTransferTest.java|   2 +-
 35 files changed, 1452 insertions(+), 907 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/CHANGES.txt
--
diff --cc CHANGES.txt
index 6234687,62a6e6f..debdd74
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -53,7 -25,9 +53,12 @@@ Merged from 2.1
   * cannot use cql since upgrading python to 2.7.11+ (CASSANDRA-11850)
   * Allow STCS-in-L0 compactions to reduce scope with LCS (CASSANDRA-12040)
  
++<<< HEAD
++===
+ 
++>>> cassandra-3.9
  3.8
+  * RTE from new CDC column breaks in flight queries (CASSANDRA-12236)
   * Fix hdr logging for single operation workloads (CASSANDRA-12145)
   * Fix SASI PREFIX search in CONTAINS mode with partial terms 
(CASSANDRA-12073)
   * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/NEWS.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/src/java/org/apache/cassandra/db/SystemKeyspace.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/src/java/org/apache/cassandra/db/rows/Row.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/src/java/org/apache/cassandra/db/rows/Rows.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e21f726/test/unit/org/apache/cassandra/UpdateBuilder.java

[09/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java 
b/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
index 2de671c..72441cd 100644
--- a/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
+++ b/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.cql3.ColumnIdentifier;
 import org.apache.cassandra.cql3.FieldIdentifier;
 import org.apache.cassandra.cql3.functions.*;
 import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.index.TargetParser;
 import org.apache.cassandra.thrift.ThriftConversion;
@@ -564,35 +565,40 @@ public class LegacySchemaMigratorTest
 setLegacyIndexStatus(keyspace);
 }
 
-private static Mutation makeLegacyCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+private static DecoratedKey decorate(CFMetaData metadata, Object value)
 {
-// Note that because Keyspaces is a COMPACT TABLE, we're really only 
setting static columns internally and shouldn't set any clustering.
-RowUpdateBuilder adder = new 
RowUpdateBuilder(SystemKeyspace.LegacyKeyspaces, timestamp, keyspace.name);
+return 
metadata.decorateKey(((AbstractType)metadata.getKeyValidator()).decompose(value));
+}
 
-adder.add("durable_writes", keyspace.params.durableWrites)
- .add("strategy_class", 
keyspace.params.replication.klass.getName())
- .add("strategy_options", 
json(keyspace.params.replication.options));
+private static Mutation makeLegacyCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+{
+Mutation.SimpleBuilder builder = 
Mutation.simpleBuilder(SystemKeyspace.NAME, 
decorate(SystemKeyspace.LegacyKeyspaces, keyspace.name))
+ .timestamp(timestamp);
 
-Mutation mutation = adder.build();
+builder.update(SystemKeyspace.LegacyKeyspaces)
+   .row()
+   .add("durable_writes", keyspace.params.durableWrites)
+   .add("strategy_class", 
keyspace.params.replication.klass.getName())
+   .add("strategy_options", 
json(keyspace.params.replication.options));
 
-keyspace.tables.forEach(table -> addTableToSchemaMutation(table, 
timestamp, true, mutation));
-keyspace.types.forEach(type -> addTypeToSchemaMutation(type, 
timestamp, mutation));
-keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, timestamp, mutation));
-keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, timestamp, mutation));
+keyspace.tables.forEach(table -> addTableToSchemaMutation(table, true, 
builder));
+keyspace.types.forEach(type -> addTypeToSchemaMutation(type, builder));
+keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, builder));
+keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, builder));
 
-return mutation;
+return builder.build();
 }
 
 /*
  * Serializing tables
  */
 
-private static void addTableToSchemaMutation(CFMetaData table, long 
timestamp, boolean withColumnsAndTriggers, Mutation mutation)
+private static void addTableToSchemaMutation(CFMetaData table, boolean 
withColumnsAndTriggers, Mutation.SimpleBuilder builder)
 {
 // For property that can be null (and can be changed), we insert 
tombstones, to make sure
 // we don't keep a property the user has removed
-RowUpdateBuilder adder = new 
RowUpdateBuilder(SystemKeyspace.LegacyColumnfamilies, timestamp, mutation)
- .clustering(table.cfName);
+Row.SimpleBuilder adder = 
builder.update(SystemKeyspace.LegacyColumnfamilies)
+ .row(table.cfName);
 
 adder.add("cf_id", table.cfId)
  .add("type", table.isSuper() ? "Super" : "Standard");
@@ -625,12 +631,14 @@ public class LegacySchemaMigratorTest
  .add("read_repair_chance", table.params.readRepairChance)
  .add("speculative_retry", 
table.params.speculativeRetry.toString());
 
+Map dropped = new HashMap<>();
 for (Map.Entry entry : 
table.getDroppedColumns().entrySet())
 {
 String name = UTF8Type.instance.getString(entry.getKey());
 CFMetaData.DroppedColumn column = entry.getValue();
-adder.addMapEntry("dropped_columns", name, column.droppedTime);
+dropped.put(name, column.droppedTime);
 }
+adder.add("dropped_columns", 

[01/14] cassandra git commit: Update build.xml and CHANGES.txt for 3.8

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.8 b27e2f93c -> 26838063d
  refs/heads/cassandra-3.9 1323ad0fb -> b603720e4
  refs/heads/trunk 5cc0d152b -> 4e21f7267


Update build.xml and CHANGES.txt for 3.8


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c3ded055
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c3ded055
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c3ded055

Branch: refs/heads/cassandra-3.9
Commit: c3ded0551f538f7845602b27d53240cd8129265c
Parents: 2aa7663
Author: Aleksey Yeschenko 
Authored: Mon Jul 18 16:47:52 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Mon Jul 18 16:47:52 2016 +0100

--
 CHANGES.txt | 48 +---
 build.xml   |  2 +-
 2 files changed, 22 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c3ded055/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3307fb3..4330fde 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,33 +1,7 @@
-3.9
+3.8
  * Fix hdr logging for single operation workloads (CASSANDRA-12145)
  * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)
-Merged from 3.0:
- * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
- * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
- * Fix migration of static thrift column names with non-text comparators 
(CASSANDRA-12147)
- * Fix upgrading sparse tables that are incorrectly marked as dense 
(CASSANDRA-11315)
- * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)
- * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098)
- * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996)
- * Always select the live sstables when getting sstables in bounds 
(CASSANDRA-11944)
- * Fix column ordering of results with static columns for Thrift requests in
-   a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of
-   those static columns in query results (CASSANDRA-12123)
- * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
- * Fix EOF exception when altering column type (CASSANDRA-11820)
-Merged from 2.2:
- * Synchronize ThriftServer::stop() (CASSANDRA-12105)
- * Use dedicated thread for JMX notifications (CASSANDRA-12146)
- * Improve streaming synchronization and fault tolerance (CASSANDRA-11414)
- * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
-Merged from 2.1:
- * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
- * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
- * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
-
-
-3.8
  * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
  * Upgrade netty to 4.0.39 (CASSANDRA-12032, CASSANDRA-12034)
  * Improve details in compaction log message (CASSANDRA-12080)
@@ -53,18 +27,38 @@ Merged from 2.1:
  * Add repaired percentage metric (CASSANDRA-11503)
  * Add Change-Data-Capture (CASSANDRA-8844)
 Merged from 3.0:
+ * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
+ * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
+ * Fix migration of static thrift column names with non-text comparators 
(CASSANDRA-12147)
+ * Fix upgrading sparse tables that are incorrectly marked as dense 
(CASSANDRA-11315)
+ * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)
+ * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098)
+ * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996)
+ * Always select the live sstables when getting sstables in bounds 
(CASSANDRA-11944)
+ * Fix column ordering of results with static columns for Thrift requests in
+   a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of
+   those static columns in query results (CASSANDRA-12123)
+ * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
+ * Fix EOF exception when altering column type (CASSANDRA-11820)
  * cqlsh: fix error handling in rare COPY FROM failure scenario 
(CASSANDRA-12070)
  * Disable autocompaction during drain (CASSANDRA-11878)
  * Add a metrics timer to MemtablePool and use it to track time spent blocked 
on memory in MemtableAllocator (CASSANDRA-11327)
  * Fix upgrading schema with super columns with non-text subcomparators 
(CASSANDRA-12023)
  * 

[12/14] cassandra git commit: Merge branch 'cassandra-3.8' into cassandra-3.9

[slebresne]

Merge branch 'cassandra-3.8' into cassandra-3.9

* cassandra-3.8:
  Fix RTE on mixed-version cluster due to CDC schema changes.
  c* uses commons-lang3, not commons-lang
  Update build.xml and CHANGES.txt for 3.8


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b603720e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b603720e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b603720e

Branch: refs/heads/trunk
Commit: b603720e4e0e9a9ed5f077e3a0f8819ef2d11441
Parents: 1323ad0 2683806
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:42:06 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:42:06 2016 +0200

--
 CHANGES.txt |   1 +
 NEWS.txt|  10 +-
 build.xml   |   2 +-
 .../cassandra/batchlog/BatchlogManager.java |  19 +-
 .../batchlog/LegacyBatchlogMigrator.java|   9 +-
 src/java/org/apache/cassandra/db/Mutation.java  |  66 +++
 .../apache/cassandra/db/RowUpdateBuilder.java   | 400 
 .../org/apache/cassandra/db/SimpleBuilders.java | 461 +++
 .../org/apache/cassandra/db/SystemKeyspace.java |  11 +-
 .../cassandra/db/commitlog/CommitLogReader.java |   2 +-
 .../db/partitions/AbstractBTreePartition.java   |   2 +-
 .../db/partitions/PartitionUpdate.java  | 154 +++
 src/java/org/apache/cassandra/db/rows/Row.java  |  99 
 src/java/org/apache/cassandra/db/rows/Rows.java |  16 +
 .../apache/cassandra/db/transform/BaseRows.java |   3 +-
 .../cassandra/schema/LegacySchemaMigrator.java  |  12 +-
 .../apache/cassandra/schema/SchemaKeyspace.java | 427 +
 .../cassandra/service/MigrationManager.java |   8 +-
 .../apache/cassandra/tracing/TraceKeyspace.java |  52 ++-
 .../org/apache/cassandra/UpdateBuilder.java |  56 +--
 test/unit/org/apache/cassandra/Util.java|  26 +-
 .../apache/cassandra/batchlog/BatchTest.java|  17 +-
 .../apache/cassandra/config/CFMetaDataTest.java |   2 +-
 .../apache/cassandra/cql3/CDCStatementTest.java |  10 +
 .../entities/RowUpdateBuilderTest.java  |  79 
 .../db/RecoveryManagerMissingHeaderTest.java|   4 +-
 .../cassandra/db/RecoveryManagerTest.java   |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   | 196 
 .../cassandra/db/compaction/TTLExpiryTest.java  |   2 +-
 .../db/partition/PartitionUpdateTest.java   |  23 +-
 .../org/apache/cassandra/hints/HintTest.java|  56 +--
 .../hints/LegacyHintsMigratorTest.java  |   3 +-
 .../org/apache/cassandra/schema/DefsTest.java   |   2 +-
 .../schema/LegacySchemaMigratorTest.java| 111 ++---
 .../cassandra/schema/SchemaKeyspaceTest.java|   6 +-
 .../cassandra/service/DataResolverTest.java |   2 +-
 .../streaming/StreamingTransferTest.java|   2 +-
 37 files changed, 1450 insertions(+), 909 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b603720e/CHANGES.txt
--
diff --cc CHANGES.txt
index 0d79a9f,388a290..62a6e6f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,32 -1,5 +1,33 @@@
 +3.9
 + * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * cqlsh: Fix handling of $$-escaped strings (CASSANDRA-12189)
 + * Fix SSL JMX requiring truststore containing server cert (CASSANDRA-12109)
 +Merged from 3.0:
 + * Fix upgrade of super columns on thrift (CASSANDRA-12335)
 + * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
 + * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
 + * Exception when computing read-repair for range tombstones (CASSANDRA-12263)
 + * Lost counter writes in compact table and static columns (CASSANDRA-12219)
 + * AssertionError with MVs on updating a row that isn't indexed due to a null 
value (CASSANDRA-12247)
 + * Disable RR and speculative retry with EACH_QUORUM reads (CASSANDRA-11980)
 + * Add option to override compaction space check (CASSANDRA-12180)
 + * Faster startup by only scanning each directory for temporary files once 
(CASSANDRA-12114)
 + * Respond with v1/v2 protocol header when responding to driver that attempts
 +   to connect with too low of a protocol version (CASSANDRA-11464)
 + * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
 + * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 +Merged from 2.2:
 + * Wait for tracing events before returning response and query at same 
consistency level client 

[11/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

Fix RTE on mixed-version cluster due to CDC schema changes.

Patch by jmckenzie and slebresne; reviewed by ayeschenko for CASSANDRA-12236


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/26838063
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/26838063
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/26838063

Branch: refs/heads/trunk
Commit: 26838063de6246e3a1e18062114ca92fb81c00cf
Parents: b27e2f9
Author: Josh McKenzie 
Authored: Thu Jul 21 12:45:13 2016 -0400
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:41:24 2016 +0200

--
 CHANGES.txt |   1 +
 NEWS.txt|  10 +-
 .../cassandra/batchlog/BatchlogManager.java |  19 +-
 .../batchlog/LegacyBatchlogMigrator.java|   9 +-
 src/java/org/apache/cassandra/db/Mutation.java  |  66 +++
 .../apache/cassandra/db/RowUpdateBuilder.java   | 400 
 .../org/apache/cassandra/db/SimpleBuilders.java | 461 +++
 .../org/apache/cassandra/db/SystemKeyspace.java |  11 +-
 .../db/partitions/AbstractBTreePartition.java   |   2 +-
 .../db/partitions/PartitionUpdate.java  | 154 +++
 src/java/org/apache/cassandra/db/rows/Row.java  |  99 
 src/java/org/apache/cassandra/db/rows/Rows.java |  16 +
 .../apache/cassandra/db/transform/BaseRows.java |   3 +-
 .../cassandra/schema/LegacySchemaMigrator.java  |  12 +-
 .../apache/cassandra/schema/SchemaKeyspace.java | 427 +
 .../cassandra/service/MigrationManager.java |   8 +-
 .../apache/cassandra/tracing/TraceKeyspace.java |  52 ++-
 .../org/apache/cassandra/UpdateBuilder.java |  56 +--
 test/unit/org/apache/cassandra/Util.java|  26 +-
 .../apache/cassandra/batchlog/BatchTest.java|  17 +-
 .../apache/cassandra/config/CFMetaDataTest.java |   2 +-
 .../apache/cassandra/cql3/CDCStatementTest.java |  10 +
 .../entities/RowUpdateBuilderTest.java  |  79 
 .../db/RecoveryManagerMissingHeaderTest.java|   4 +-
 .../cassandra/db/RecoveryManagerTest.java   |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   | 196 
 .../cassandra/db/compaction/TTLExpiryTest.java  |   2 +-
 .../db/partition/PartitionUpdateTest.java   |  23 +-
 .../org/apache/cassandra/hints/HintTest.java|  56 +--
 .../hints/LegacyHintsMigratorTest.java  |   3 +-
 .../org/apache/cassandra/schema/DefsTest.java   |   2 +-
 .../schema/LegacySchemaMigratorTest.java| 111 ++---
 .../cassandra/schema/SchemaKeyspaceTest.java|   6 +-
 .../cassandra/service/DataResolverTest.java |   2 +-
 .../streaming/StreamingTransferTest.java|   2 +-
 35 files changed, 1448 insertions(+), 907 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4330fde..388a290 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.8
+ * RTE from new CDC column breaks in flight queries (CASSANDRA-12236)
  * Fix hdr logging for single operation workloads (CASSANDRA-12145)
  * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 7418f3a..d8d84f5 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -39,6 +39,9 @@ New features
  the data/cdc_raw directory until removed by the user and writes to 
CDC-enabled tables
  will be rejected with a WriteTimeoutException once cdc_total_space_in_mb 
is reached
  between unflushed CommitLogSegments and cdc_raw.
+ NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on 
a mixed-version
+ cluster as it will lead to exceptions which can interrupt traffic. Once 
all nodes
+ have been upgraded to 3.8 it is safe to enable this feature and restart 
the cluster.
 
 Upgrading
 -
@@ -48,13 +51,6 @@ Upgrading
   those under a different name, change your code to use the new names and
   drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 
for more
   details).
-- Due to changes in schema migration handling and the storage format after 
3.0, you will
-  see error messages such as:
- "java.lang.RuntimeException: Unknown column cdc during 
deserialization"
-  in your system logs on a mixed-version cluster during upgrades. This 
error message
-  is harmless and due to the 3.8 nodes having cdc added to their schema 
tables while
-  the <3.8 nodes do not. This message 

[07/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index 8e3961e..7a90dab 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -266,8 +266,8 @@ public final class SchemaKeyspace
 }
 
 // (+1 to timestamp to make sure we don't get shadowed by the 
tombstones we just added)
-makeCreateKeyspaceMutation(system, timestamp + 1).apply();
-makeCreateKeyspaceMutation(schema, timestamp + 1).apply();
+makeCreateKeyspaceMutation(system, timestamp + 1).build().apply();
+makeCreateKeyspaceMutation(schema, timestamp + 1).build().apply();
 }
 
 public static void truncate()
@@ -377,11 +377,6 @@ public final class SchemaKeyspace
 }
 }
 
-private static ByteBuffer getSchemaKSKey(String ksName)
-{
-return AsciiType.instance.fromString(ksName);
-}
-
 private static boolean isSystemKeyspaceSchemaPartition(DecoratedKey 
partitionKey)
 {
 return 
Schema.isSystemKeyspace(UTF8Type.instance.compose(partitionKey.getKey()));
@@ -391,152 +386,152 @@ public final class SchemaKeyspace
  * Schema entities to mutations
  */
 
-public static Mutation makeCreateKeyspaceMutation(String name, 
KeyspaceParams params, long timestamp)
+private static DecoratedKey decorate(CFMetaData metadata, Object value)
 {
-RowUpdateBuilder adder = new RowUpdateBuilder(Keyspaces, timestamp, 
name).clustering();
-return adder.add(KeyspaceParams.Option.DURABLE_WRITES.toString(), 
params.durableWrites)
-.frozenMap(KeyspaceParams.Option.REPLICATION.toString(), 
params.replication.asMap())
-.build();
+return 
metadata.decorateKey(((AbstractType)metadata.getKeyValidator()).decompose(value));
 }
 
-public static Mutation makeCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+public static Mutation.SimpleBuilder makeCreateKeyspaceMutation(String 
name, KeyspaceParams params, long timestamp)
 {
-Mutation mutation = makeCreateKeyspaceMutation(keyspace.name, 
keyspace.params, timestamp);
+Mutation.SimpleBuilder builder = 
Mutation.simpleBuilder(Keyspaces.ksName, decorate(Keyspaces, name))
+ .timestamp(timestamp);
 
-keyspace.tables.forEach(table -> addTableToSchemaMutation(table, 
timestamp, true, mutation));
-keyspace.views.forEach(view -> addViewToSchemaMutation(view, 
timestamp, true, mutation));
-keyspace.types.forEach(type -> addTypeToSchemaMutation(type, 
timestamp, mutation));
-keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, timestamp, mutation));
-keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, timestamp, mutation));
+builder.update(Keyspaces)
+   .row()
+   .add(KeyspaceParams.Option.DURABLE_WRITES.toString(), 
params.durableWrites)
+   .add(KeyspaceParams.Option.REPLICATION.toString(), 
params.replication.asMap());
 
-return mutation;
+return builder;
 }
 
-public static Mutation makeDropKeyspaceMutation(KeyspaceMetadata keyspace, 
long timestamp)
+public static Mutation.SimpleBuilder 
makeCreateKeyspaceMutation(KeyspaceMetadata keyspace, long timestamp)
 {
-int nowInSec = FBUtilities.nowInSeconds();
-Mutation mutation = new Mutation(NAME, 
Keyspaces.decorateKey(getSchemaKSKey(keyspace.name)));
+Mutation.SimpleBuilder builder = 
makeCreateKeyspaceMutation(keyspace.name, keyspace.params, timestamp);
 
-for (CFMetaData schemaTable : ALL_TABLE_METADATA)
-mutation.add(PartitionUpdate.fullPartitionDelete(schemaTable, 
mutation.key(), timestamp, nowInSec));
+keyspace.tables.forEach(table -> addTableToSchemaMutation(table, true, 
builder));
+keyspace.views.forEach(view -> addViewToSchemaMutation(view, true, 
builder));
+keyspace.types.forEach(type -> addTypeToSchemaMutation(type, builder));
+keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, builder));
+keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, builder));
 
-return mutation;
+return builder;
 }
 
-public static Mutation makeCreateTypeMutation(KeyspaceMetadata keyspace, 
UserType type, long timestamp)
+public static Mutation.SimpleBuilder 
makeDropKeyspaceMutation(KeyspaceMetadata keyspace, long timestamp)
 {
-// Include the serialized keyspace in case the target node missed a 
CREATE KEYSPACE migration (see CASSANDRA-5631).
-Mutation 

[05/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

Fix RTE on mixed-version cluster due to CDC schema changes.

Patch by jmckenzie and slebresne; reviewed by ayeschenko for CASSANDRA-12236


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/26838063
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/26838063
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/26838063

Branch: refs/heads/cassandra-3.9
Commit: 26838063de6246e3a1e18062114ca92fb81c00cf
Parents: b27e2f9
Author: Josh McKenzie 
Authored: Thu Jul 21 12:45:13 2016 -0400
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:41:24 2016 +0200

--
 CHANGES.txt |   1 +
 NEWS.txt|  10 +-
 .../cassandra/batchlog/BatchlogManager.java |  19 +-
 .../batchlog/LegacyBatchlogMigrator.java|   9 +-
 src/java/org/apache/cassandra/db/Mutation.java  |  66 +++
 .../apache/cassandra/db/RowUpdateBuilder.java   | 400 
 .../org/apache/cassandra/db/SimpleBuilders.java | 461 +++
 .../org/apache/cassandra/db/SystemKeyspace.java |  11 +-
 .../db/partitions/AbstractBTreePartition.java   |   2 +-
 .../db/partitions/PartitionUpdate.java  | 154 +++
 src/java/org/apache/cassandra/db/rows/Row.java  |  99 
 src/java/org/apache/cassandra/db/rows/Rows.java |  16 +
 .../apache/cassandra/db/transform/BaseRows.java |   3 +-
 .../cassandra/schema/LegacySchemaMigrator.java  |  12 +-
 .../apache/cassandra/schema/SchemaKeyspace.java | 427 +
 .../cassandra/service/MigrationManager.java |   8 +-
 .../apache/cassandra/tracing/TraceKeyspace.java |  52 ++-
 .../org/apache/cassandra/UpdateBuilder.java |  56 +--
 test/unit/org/apache/cassandra/Util.java|  26 +-
 .../apache/cassandra/batchlog/BatchTest.java|  17 +-
 .../apache/cassandra/config/CFMetaDataTest.java |   2 +-
 .../apache/cassandra/cql3/CDCStatementTest.java |  10 +
 .../entities/RowUpdateBuilderTest.java  |  79 
 .../db/RecoveryManagerMissingHeaderTest.java|   4 +-
 .../cassandra/db/RecoveryManagerTest.java   |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   | 196 
 .../cassandra/db/compaction/TTLExpiryTest.java  |   2 +-
 .../db/partition/PartitionUpdateTest.java   |  23 +-
 .../org/apache/cassandra/hints/HintTest.java|  56 +--
 .../hints/LegacyHintsMigratorTest.java  |   3 +-
 .../org/apache/cassandra/schema/DefsTest.java   |   2 +-
 .../schema/LegacySchemaMigratorTest.java| 111 ++---
 .../cassandra/schema/SchemaKeyspaceTest.java|   6 +-
 .../cassandra/service/DataResolverTest.java |   2 +-
 .../streaming/StreamingTransferTest.java|   2 +-
 35 files changed, 1448 insertions(+), 907 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4330fde..388a290 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.8
+ * RTE from new CDC column breaks in flight queries (CASSANDRA-12236)
  * Fix hdr logging for single operation workloads (CASSANDRA-12145)
  * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 7418f3a..d8d84f5 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -39,6 +39,9 @@ New features
  the data/cdc_raw directory until removed by the user and writes to 
CDC-enabled tables
  will be rejected with a WriteTimeoutException once cdc_total_space_in_mb 
is reached
  between unflushed CommitLogSegments and cdc_raw.
+ NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on 
a mixed-version
+ cluster as it will lead to exceptions which can interrupt traffic. Once 
all nodes
+ have been upgraded to 3.8 it is safe to enable this feature and restart 
the cluster.
 
 Upgrading
 -
@@ -48,13 +51,6 @@ Upgrading
   those under a different name, change your code to use the new names and
   drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 
for more
   details).
-- Due to changes in schema migration handling and the storage format after 
3.0, you will
-  see error messages such as:
- "java.lang.RuntimeException: Unknown column cdc during 
deserialization"
-  in your system logs on a mixed-version cluster during upgrades. This 
error message
-  is harmless and due to the 3.8 nodes having cdc added to their schema 
tables while
-  the <3.8 nodes do not. This 

[13/14] cassandra git commit: Merge branch 'cassandra-3.8' into cassandra-3.9

[slebresne]

Merge branch 'cassandra-3.8' into cassandra-3.9

* cassandra-3.8:
  Fix RTE on mixed-version cluster due to CDC schema changes.
  c* uses commons-lang3, not commons-lang
  Update build.xml and CHANGES.txt for 3.8


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b603720e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b603720e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b603720e

Branch: refs/heads/cassandra-3.9
Commit: b603720e4e0e9a9ed5f077e3a0f8819ef2d11441
Parents: 1323ad0 2683806
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:42:06 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:42:06 2016 +0200

--
 CHANGES.txt |   1 +
 NEWS.txt|  10 +-
 build.xml   |   2 +-
 .../cassandra/batchlog/BatchlogManager.java |  19 +-
 .../batchlog/LegacyBatchlogMigrator.java|   9 +-
 src/java/org/apache/cassandra/db/Mutation.java  |  66 +++
 .../apache/cassandra/db/RowUpdateBuilder.java   | 400 
 .../org/apache/cassandra/db/SimpleBuilders.java | 461 +++
 .../org/apache/cassandra/db/SystemKeyspace.java |  11 +-
 .../cassandra/db/commitlog/CommitLogReader.java |   2 +-
 .../db/partitions/AbstractBTreePartition.java   |   2 +-
 .../db/partitions/PartitionUpdate.java  | 154 +++
 src/java/org/apache/cassandra/db/rows/Row.java  |  99 
 src/java/org/apache/cassandra/db/rows/Rows.java |  16 +
 .../apache/cassandra/db/transform/BaseRows.java |   3 +-
 .../cassandra/schema/LegacySchemaMigrator.java  |  12 +-
 .../apache/cassandra/schema/SchemaKeyspace.java | 427 +
 .../cassandra/service/MigrationManager.java |   8 +-
 .../apache/cassandra/tracing/TraceKeyspace.java |  52 ++-
 .../org/apache/cassandra/UpdateBuilder.java |  56 +--
 test/unit/org/apache/cassandra/Util.java|  26 +-
 .../apache/cassandra/batchlog/BatchTest.java|  17 +-
 .../apache/cassandra/config/CFMetaDataTest.java |   2 +-
 .../apache/cassandra/cql3/CDCStatementTest.java |  10 +
 .../entities/RowUpdateBuilderTest.java  |  79 
 .../db/RecoveryManagerMissingHeaderTest.java|   4 +-
 .../cassandra/db/RecoveryManagerTest.java   |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   | 196 
 .../cassandra/db/compaction/TTLExpiryTest.java  |   2 +-
 .../db/partition/PartitionUpdateTest.java   |  23 +-
 .../org/apache/cassandra/hints/HintTest.java|  56 +--
 .../hints/LegacyHintsMigratorTest.java  |   3 +-
 .../org/apache/cassandra/schema/DefsTest.java   |   2 +-
 .../schema/LegacySchemaMigratorTest.java| 111 ++---
 .../cassandra/schema/SchemaKeyspaceTest.java|   6 +-
 .../cassandra/service/DataResolverTest.java |   2 +-
 .../streaming/StreamingTransferTest.java|   2 +-
 37 files changed, 1450 insertions(+), 909 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b603720e/CHANGES.txt
--
diff --cc CHANGES.txt
index 0d79a9f,388a290..62a6e6f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,32 -1,5 +1,33 @@@
 +3.9
 + * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * cqlsh: Fix handling of $$-escaped strings (CASSANDRA-12189)
 + * Fix SSL JMX requiring truststore containing server cert (CASSANDRA-12109)
 +Merged from 3.0:
 + * Fix upgrade of super columns on thrift (CASSANDRA-12335)
 + * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
 + * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
 + * Exception when computing read-repair for range tombstones (CASSANDRA-12263)
 + * Lost counter writes in compact table and static columns (CASSANDRA-12219)
 + * AssertionError with MVs on updating a row that isn't indexed due to a null 
value (CASSANDRA-12247)
 + * Disable RR and speculative retry with EACH_QUORUM reads (CASSANDRA-11980)
 + * Add option to override compaction space check (CASSANDRA-12180)
 + * Faster startup by only scanning each directory for temporary files once 
(CASSANDRA-12114)
 + * Respond with v1/v2 protocol header when responding to driver that attempts
 +   to connect with too low of a protocol version (CASSANDRA-11464)
 + * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
 + * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 +Merged from 2.2:
 + * Wait for tracing events before returning response and query at same 
consistency level 

[06/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java 
b/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
index 2de671c..72441cd 100644
--- a/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
+++ b/test/unit/org/apache/cassandra/schema/LegacySchemaMigratorTest.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.cql3.ColumnIdentifier;
 import org.apache.cassandra.cql3.FieldIdentifier;
 import org.apache.cassandra.cql3.functions.*;
 import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.index.TargetParser;
 import org.apache.cassandra.thrift.ThriftConversion;
@@ -564,35 +565,40 @@ public class LegacySchemaMigratorTest
 setLegacyIndexStatus(keyspace);
 }
 
-private static Mutation makeLegacyCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+private static DecoratedKey decorate(CFMetaData metadata, Object value)
 {
-// Note that because Keyspaces is a COMPACT TABLE, we're really only 
setting static columns internally and shouldn't set any clustering.
-RowUpdateBuilder adder = new 
RowUpdateBuilder(SystemKeyspace.LegacyKeyspaces, timestamp, keyspace.name);
+return 
metadata.decorateKey(((AbstractType)metadata.getKeyValidator()).decompose(value));
+}
 
-adder.add("durable_writes", keyspace.params.durableWrites)
- .add("strategy_class", 
keyspace.params.replication.klass.getName())
- .add("strategy_options", 
json(keyspace.params.replication.options));
+private static Mutation makeLegacyCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+{
+Mutation.SimpleBuilder builder = 
Mutation.simpleBuilder(SystemKeyspace.NAME, 
decorate(SystemKeyspace.LegacyKeyspaces, keyspace.name))
+ .timestamp(timestamp);
 
-Mutation mutation = adder.build();
+builder.update(SystemKeyspace.LegacyKeyspaces)
+   .row()
+   .add("durable_writes", keyspace.params.durableWrites)
+   .add("strategy_class", 
keyspace.params.replication.klass.getName())
+   .add("strategy_options", 
json(keyspace.params.replication.options));
 
-keyspace.tables.forEach(table -> addTableToSchemaMutation(table, 
timestamp, true, mutation));
-keyspace.types.forEach(type -> addTypeToSchemaMutation(type, 
timestamp, mutation));
-keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, timestamp, mutation));
-keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, timestamp, mutation));
+keyspace.tables.forEach(table -> addTableToSchemaMutation(table, true, 
builder));
+keyspace.types.forEach(type -> addTypeToSchemaMutation(type, builder));
+keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, builder));
+keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, builder));
 
-return mutation;
+return builder.build();
 }
 
 /*
  * Serializing tables
  */
 
-private static void addTableToSchemaMutation(CFMetaData table, long 
timestamp, boolean withColumnsAndTriggers, Mutation mutation)
+private static void addTableToSchemaMutation(CFMetaData table, boolean 
withColumnsAndTriggers, Mutation.SimpleBuilder builder)
 {
 // For property that can be null (and can be changed), we insert 
tombstones, to make sure
 // we don't keep a property the user has removed
-RowUpdateBuilder adder = new 
RowUpdateBuilder(SystemKeyspace.LegacyColumnfamilies, timestamp, mutation)
- .clustering(table.cfName);
+Row.SimpleBuilder adder = 
builder.update(SystemKeyspace.LegacyColumnfamilies)
+ .row(table.cfName);
 
 adder.add("cf_id", table.cfId)
  .add("type", table.isSuper() ? "Super" : "Standard");
@@ -625,12 +631,14 @@ public class LegacySchemaMigratorTest
  .add("read_repair_chance", table.params.readRepairChance)
  .add("speculative_retry", 
table.params.speculativeRetry.toString());
 
+Map dropped = new HashMap<>();
 for (Map.Entry entry : 
table.getDroppedColumns().entrySet())
 {
 String name = UTF8Type.instance.getString(entry.getKey());
 CFMetaData.DroppedColumn column = entry.getValue();
-adder.addMapEntry("dropped_columns", name, column.droppedTime);
+dropped.put(name, column.droppedTime);
 }
+adder.add("dropped_columns", 

[04/14] cassandra git commit: Fix RTE on mixed-version cluster due to CDC schema changes.

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/26838063/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index 8e3961e..7a90dab 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -266,8 +266,8 @@ public final class SchemaKeyspace
 }
 
 // (+1 to timestamp to make sure we don't get shadowed by the 
tombstones we just added)
-makeCreateKeyspaceMutation(system, timestamp + 1).apply();
-makeCreateKeyspaceMutation(schema, timestamp + 1).apply();
+makeCreateKeyspaceMutation(system, timestamp + 1).build().apply();
+makeCreateKeyspaceMutation(schema, timestamp + 1).build().apply();
 }
 
 public static void truncate()
@@ -377,11 +377,6 @@ public final class SchemaKeyspace
 }
 }
 
-private static ByteBuffer getSchemaKSKey(String ksName)
-{
-return AsciiType.instance.fromString(ksName);
-}
-
 private static boolean isSystemKeyspaceSchemaPartition(DecoratedKey 
partitionKey)
 {
 return 
Schema.isSystemKeyspace(UTF8Type.instance.compose(partitionKey.getKey()));
@@ -391,152 +386,152 @@ public final class SchemaKeyspace
  * Schema entities to mutations
  */
 
-public static Mutation makeCreateKeyspaceMutation(String name, 
KeyspaceParams params, long timestamp)
+private static DecoratedKey decorate(CFMetaData metadata, Object value)
 {
-RowUpdateBuilder adder = new RowUpdateBuilder(Keyspaces, timestamp, 
name).clustering();
-return adder.add(KeyspaceParams.Option.DURABLE_WRITES.toString(), 
params.durableWrites)
-.frozenMap(KeyspaceParams.Option.REPLICATION.toString(), 
params.replication.asMap())
-.build();
+return 
metadata.decorateKey(((AbstractType)metadata.getKeyValidator()).decompose(value));
 }
 
-public static Mutation makeCreateKeyspaceMutation(KeyspaceMetadata 
keyspace, long timestamp)
+public static Mutation.SimpleBuilder makeCreateKeyspaceMutation(String 
name, KeyspaceParams params, long timestamp)
 {
-Mutation mutation = makeCreateKeyspaceMutation(keyspace.name, 
keyspace.params, timestamp);
+Mutation.SimpleBuilder builder = 
Mutation.simpleBuilder(Keyspaces.ksName, decorate(Keyspaces, name))
+ .timestamp(timestamp);
 
-keyspace.tables.forEach(table -> addTableToSchemaMutation(table, 
timestamp, true, mutation));
-keyspace.views.forEach(view -> addViewToSchemaMutation(view, 
timestamp, true, mutation));
-keyspace.types.forEach(type -> addTypeToSchemaMutation(type, 
timestamp, mutation));
-keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, timestamp, mutation));
-keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, timestamp, mutation));
+builder.update(Keyspaces)
+   .row()
+   .add(KeyspaceParams.Option.DURABLE_WRITES.toString(), 
params.durableWrites)
+   .add(KeyspaceParams.Option.REPLICATION.toString(), 
params.replication.asMap());
 
-return mutation;
+return builder;
 }
 
-public static Mutation makeDropKeyspaceMutation(KeyspaceMetadata keyspace, 
long timestamp)
+public static Mutation.SimpleBuilder 
makeCreateKeyspaceMutation(KeyspaceMetadata keyspace, long timestamp)
 {
-int nowInSec = FBUtilities.nowInSeconds();
-Mutation mutation = new Mutation(NAME, 
Keyspaces.decorateKey(getSchemaKSKey(keyspace.name)));
+Mutation.SimpleBuilder builder = 
makeCreateKeyspaceMutation(keyspace.name, keyspace.params, timestamp);
 
-for (CFMetaData schemaTable : ALL_TABLE_METADATA)
-mutation.add(PartitionUpdate.fullPartitionDelete(schemaTable, 
mutation.key(), timestamp, nowInSec));
+keyspace.tables.forEach(table -> addTableToSchemaMutation(table, true, 
builder));
+keyspace.views.forEach(view -> addViewToSchemaMutation(view, true, 
builder));
+keyspace.types.forEach(type -> addTypeToSchemaMutation(type, builder));
+keyspace.functions.udfs().forEach(udf -> 
addFunctionToSchemaMutation(udf, builder));
+keyspace.functions.udas().forEach(uda -> 
addAggregateToSchemaMutation(uda, builder));
 
-return mutation;
+return builder;
 }
 
-public static Mutation makeCreateTypeMutation(KeyspaceMetadata keyspace, 
UserType type, long timestamp)
+public static Mutation.SimpleBuilder 
makeDropKeyspaceMutation(KeyspaceMetadata keyspace, long timestamp)
 {
-// Include the serialized keyspace in case the target node missed a 
CREATE KEYSPACE migration (see CASSANDRA-5631).
-Mutation 

[02/14] cassandra git commit: c* uses commons-lang3, not commons-lang

[slebresne]

c* uses commons-lang3, not commons-lang


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b27e2f93
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b27e2f93
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b27e2f93

Branch: refs/heads/cassandra-3.9
Commit: b27e2f93cc9bc33a95d531f43442b93e85ba4a30
Parents: c3ded05
Author: Dave Brosius 
Authored: Mon Jul 4 17:23:46 2016 -0400
Committer: Dave Brosius 
Committed: Tue Jul 26 20:04:42 2016 -0400

--
 src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b27e2f93/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
index a914cc9..6acbd0d 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
@@ -23,7 +23,7 @@ import java.util.concurrent.atomic.AtomicInteger;
 import java.util.zip.CRC32;
 
 import com.google.common.annotations.VisibleForTesting;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 

[jira] [Resolved] (CASSANDRA-12335) Super columns are broken after upgrading to 3.0 on thrift

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-12335.
--
   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 3.x)
   3.9
   3.0.9

Alright then, committed that fix and created CASSANDRA-12373 for the CQL 
followup.

> Super columns are broken after upgrading to 3.0 on thrift
> -
>
> Key: CASSANDRA-12335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12335
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Sylvain Lebresne
> Fix For: 3.0.9, 3.9
>
> Attachments: 0001-Force-super-column-families-to-be-compound.txt
>
>
> Super Columns are broken after upgrading to cassandra-3.0 HEAD.  The below 
> script shows this.
> 2.1 cli output for get:
> {code}
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> => (name=name, value=Bob, timestamp=1469724504357000)
> {code}
> cqlsh:
> {code}
> [default@test]
>  key  | blobAsText(column1)
> --+-
>  0x53696d6f6e |attr
>  0x426f62 |attr
> {code}
> 3.0 cli:
> {code}
> [default@unknown] use test;
> unconfigured table schema_columnfamilies
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> null
> [default@test]
> {code}
> cqlsh:
> {code}
>  key  | system.blobastext(column1)
> --+--
>  0x53696d6f6e | \x00\x04attr\x00\x00\x04name\x00
>  0x426f62 | \x00\x04attr\x00\x00\x04name\x00
> {code}
> Run this from a directory with cassandra-3.0 checked out and compiled
> {code}
> ccm create -n 2 -v 2.1.14 testsuper
> echo "### Starting 2.1 ###"
> ccm start
> MYFILE=`mktemp`
> echo "create keyspace test with placement_strategy = 
> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = 
> {replication_factor:2};
> use test;
> create column family Sites with column_type = 'Super' and comparator = 
> 'BytesType' and subcomparator='UTF8Type';
> set Sites[utf8('Simon')][utf8('attr')]['name'] = utf8('Simon');
> set Sites[utf8('Bob')][utf8('attr')]['name'] = utf8('Bob');
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7100 flush
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7200 flush
> ccm stop
> # run from cassandra-3.0 checked out and compiled
> ccm setdir
> echo "### Starting Current Directory 
> ###"
> ccm start
> ./bin/nodetool -p 7100 upgradesstables
> ./bin/nodetool -p 7200 upgradesstables
> ./bin/nodetool -p 7100 enablethrift
> ./bin/nodetool -p 7200 enablethrift
> MYFILE=`mktemp`
> echo "use test;
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/6] cassandra git commit: Super columns are broken after upgrading to 3.0 on thrift

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 f5c9d6e49 -> 3431934b4
  refs/heads/cassandra-3.9 717c6d6bf -> 1323ad0fb
  refs/heads/trunk a69d13ba8 -> 5cc0d152b


Super columns are broken after upgrading to 3.0 on thrift

patch by Sylvain Lebresne; reviewed by Aleksey Yeschenko for CASSANDRA-12335


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3431934b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3431934b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3431934b

Branch: refs/heads/cassandra-3.0
Commit: 3431934b4d97472882fd90d112ec5db1bf757b0d
Parents: f5c9d6e
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:25:03 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:25:03 2016 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3431934b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d78345a..e8b8abb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.9
+ * Fix upgrade of super columns on thrift (CASSANDRA-12335)
  * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
  * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
  * Exception when computing read-repair for range tombstones (CASSANDRA-12263)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3431934b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--
diff --git a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java 
b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
index b6d8d2b..09af763 100644
--- a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
+++ b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
@@ -284,7 +284,7 @@ public final class LegacySchemaMigrator
 AbstractType subComparator = tableRow.has("subcomparator") ? 
TypeParser.parse(tableRow.getString("subcomparator")) : null;
 
 boolean isSuper = 
"super".equals(tableRow.getString("type").toLowerCase());
-boolean isCompound = rawComparator instanceof CompositeType;
+boolean isCompound = rawComparator instanceof CompositeType || isSuper;
 
 /*
  * Determine whether or not the table is *really* dense

[3/6] cassandra git commit: Super columns are broken after upgrading to 3.0 on thrift

[slebresne]

Super columns are broken after upgrading to 3.0 on thrift

patch by Sylvain Lebresne; reviewed by Aleksey Yeschenko for CASSANDRA-12335


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3431934b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3431934b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3431934b

Branch: refs/heads/trunk
Commit: 3431934b4d97472882fd90d112ec5db1bf757b0d
Parents: f5c9d6e
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:25:03 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:25:03 2016 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3431934b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d78345a..e8b8abb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.9
+ * Fix upgrade of super columns on thrift (CASSANDRA-12335)
  * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
  * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
  * Exception when computing read-repair for range tombstones (CASSANDRA-12263)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3431934b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--
diff --git a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java 
b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
index b6d8d2b..09af763 100644
--- a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
+++ b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
@@ -284,7 +284,7 @@ public final class LegacySchemaMigrator
 AbstractType subComparator = tableRow.has("subcomparator") ? 
TypeParser.parse(tableRow.getString("subcomparator")) : null;
 
 boolean isSuper = 
"super".equals(tableRow.getString("type").toLowerCase());
-boolean isCompound = rawComparator instanceof CompositeType;
+boolean isCompound = rawComparator instanceof CompositeType || isSuper;
 
 /*
  * Determine whether or not the table is *really* dense

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.9

* cassandra-3.0:
  Super columns are broken after upgrading to 3.0 on thrift


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1323ad0f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1323ad0f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1323ad0f

Branch: refs/heads/cassandra-3.9
Commit: 1323ad0fba0063b646eb0be5843956dd345159a4
Parents: 717c6d6 3431934
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:29:22 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:29:22 2016 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1323ad0f/CHANGES.txt
--
diff --cc CHANGES.txt
index bb58e77,e8b8abb..0d79a9f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,9 -1,5 +1,10 @@@
 -3.0.9
 +3.9
 + * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * cqlsh: Fix handling of $$-escaped strings (CASSANDRA-12189)
 + * Fix SSL JMX requiring truststore containing server cert (CASSANDRA-12109)
 +Merged from 3.0:
+  * Fix upgrade of super columns on thrift (CASSANDRA-12335)
   * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
   * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
   * Exception when computing read-repair for range tombstones (CASSANDRA-12263)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1323ad0f/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.9

* cassandra-3.0:
  Super columns are broken after upgrading to 3.0 on thrift


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1323ad0f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1323ad0f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1323ad0f

Branch: refs/heads/trunk
Commit: 1323ad0fba0063b646eb0be5843956dd345159a4
Parents: 717c6d6 3431934
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:29:22 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:29:22 2016 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1323ad0f/CHANGES.txt
--
diff --cc CHANGES.txt
index bb58e77,e8b8abb..0d79a9f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,9 -1,5 +1,10 @@@
 -3.0.9
 +3.9
 + * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * cqlsh: Fix handling of $$-escaped strings (CASSANDRA-12189)
 + * Fix SSL JMX requiring truststore containing server cert (CASSANDRA-12109)
 +Merged from 3.0:
+  * Fix upgrade of super columns on thrift (CASSANDRA-12335)
   * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
   * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
   * Exception when computing read-repair for range tombstones (CASSANDRA-12263)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1323ad0f/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--

[6/6] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[slebresne]

Merge branch 'cassandra-3.9' into trunk

* cassandra-3.9:
  Super columns are broken after upgrading to 3.0 on thrift


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5cc0d152
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5cc0d152
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5cc0d152

Branch: refs/heads/trunk
Commit: 5cc0d152bde6eb027bf7c1e86f0bc4256e688daa
Parents: a69d13b 1323ad0
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:29:44 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:29:44 2016 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5cc0d152/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5cc0d152/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--

[2/6] cassandra git commit: Super columns are broken after upgrading to 3.0 on thrift

[slebresne]

Super columns are broken after upgrading to 3.0 on thrift

patch by Sylvain Lebresne; reviewed by Aleksey Yeschenko for CASSANDRA-12335


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3431934b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3431934b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3431934b

Branch: refs/heads/cassandra-3.9
Commit: 3431934b4d97472882fd90d112ec5db1bf757b0d
Parents: f5c9d6e
Author: Sylvain Lebresne 
Authored: Wed Aug 3 17:25:03 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Aug 3 17:25:03 2016 +0200

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3431934b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d78345a..e8b8abb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.9
+ * Fix upgrade of super columns on thrift (CASSANDRA-12335)
  * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
  * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
  * Exception when computing read-repair for range tombstones (CASSANDRA-12263)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3431934b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--
diff --git a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java 
b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
index b6d8d2b..09af763 100644
--- a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
+++ b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
@@ -284,7 +284,7 @@ public final class LegacySchemaMigrator
 AbstractType subComparator = tableRow.has("subcomparator") ? 
TypeParser.parse(tableRow.getString("subcomparator")) : null;
 
 boolean isSuper = 
"super".equals(tableRow.getString("type").toLowerCase());
-boolean isCompound = rawComparator instanceof CompositeType;
+boolean isCompound = rawComparator instanceof CompositeType || isSuper;
 
 /*
  * Determine whether or not the table is *really* dense

[jira] [Updated] (CASSANDRA-12335) Super columns are broken after upgrading to 3.0 on thrift

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-12335:
-
Reviewer: Aleksey Yeschenko

> Super columns are broken after upgrading to 3.0 on thrift
> -
>
> Key: CASSANDRA-12335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12335
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-Force-super-column-families-to-be-compound.txt
>
>
> Super Columns are broken after upgrading to cassandra-3.0 HEAD.  The below 
> script shows this.
> 2.1 cli output for get:
> {code}
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> => (name=name, value=Bob, timestamp=1469724504357000)
> {code}
> cqlsh:
> {code}
> [default@test]
>  key  | blobAsText(column1)
> --+-
>  0x53696d6f6e |attr
>  0x426f62 |attr
> {code}
> 3.0 cli:
> {code}
> [default@unknown] use test;
> unconfigured table schema_columnfamilies
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> null
> [default@test]
> {code}
> cqlsh:
> {code}
>  key  | system.blobastext(column1)
> --+--
>  0x53696d6f6e | \x00\x04attr\x00\x00\x04name\x00
>  0x426f62 | \x00\x04attr\x00\x00\x04name\x00
> {code}
> Run this from a directory with cassandra-3.0 checked out and compiled
> {code}
> ccm create -n 2 -v 2.1.14 testsuper
> echo "### Starting 2.1 ###"
> ccm start
> MYFILE=`mktemp`
> echo "create keyspace test with placement_strategy = 
> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = 
> {replication_factor:2};
> use test;
> create column family Sites with column_type = 'Super' and comparator = 
> 'BytesType' and subcomparator='UTF8Type';
> set Sites[utf8('Simon')][utf8('attr')]['name'] = utf8('Simon');
> set Sites[utf8('Bob')][utf8('attr')]['name'] = utf8('Bob');
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7100 flush
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7200 flush
> ccm stop
> # run from cassandra-3.0 checked out and compiled
> ccm setdir
> echo "### Starting Current Directory 
> ###"
> ccm start
> ./bin/nodetool -p 7100 upgradesstables
> ./bin/nodetool -p 7200 upgradesstables
> ./bin/nodetool -p 7100 enablethrift
> ./bin/nodetool -p 7200 enablethrift
> MYFILE=`mktemp`
> echo "use test;
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12335) Super columns are broken after upgrading to 3.0 on thrift

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne reassigned CASSANDRA-12335:


Assignee: Sylvain Lebresne

> Super columns are broken after upgrading to 3.0 on thrift
> -
>
> Key: CASSANDRA-12335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12335
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-Force-super-column-families-to-be-compound.txt
>
>
> Super Columns are broken after upgrading to cassandra-3.0 HEAD.  The below 
> script shows this.
> 2.1 cli output for get:
> {code}
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> => (name=name, value=Bob, timestamp=1469724504357000)
> {code}
> cqlsh:
> {code}
> [default@test]
>  key  | blobAsText(column1)
> --+-
>  0x53696d6f6e |attr
>  0x426f62 |attr
> {code}
> 3.0 cli:
> {code}
> [default@unknown] use test;
> unconfigured table schema_columnfamilies
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> null
> [default@test]
> {code}
> cqlsh:
> {code}
>  key  | system.blobastext(column1)
> --+--
>  0x53696d6f6e | \x00\x04attr\x00\x00\x04name\x00
>  0x426f62 | \x00\x04attr\x00\x00\x04name\x00
> {code}
> Run this from a directory with cassandra-3.0 checked out and compiled
> {code}
> ccm create -n 2 -v 2.1.14 testsuper
> echo "### Starting 2.1 ###"
> ccm start
> MYFILE=`mktemp`
> echo "create keyspace test with placement_strategy = 
> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = 
> {replication_factor:2};
> use test;
> create column family Sites with column_type = 'Super' and comparator = 
> 'BytesType' and subcomparator='UTF8Type';
> set Sites[utf8('Simon')][utf8('attr')]['name'] = utf8('Simon');
> set Sites[utf8('Bob')][utf8('attr')]['name'] = utf8('Bob');
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7100 flush
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7200 flush
> ccm stop
> # run from cassandra-3.0 checked out and compiled
> ccm setdir
> echo "### Starting Current Directory 
> ###"
> ccm start
> ./bin/nodetool -p 7100 upgradesstables
> ./bin/nodetool -p 7200 upgradesstables
> ./bin/nodetool -p 7100 enablethrift
> ./bin/nodetool -p 7200 enablethrift
> MYFILE=`mktemp`
> echo "use test;
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12335) Super columns are broken after upgrading to 3.0 on thrift

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-12335:
-
Summary: Super columns are broken after upgrading to 3.0 on thrift  (was: 
Super columns are broken after upgrading to 3.0)

> Super columns are broken after upgrading to 3.0 on thrift
> -
>
> Key: CASSANDRA-12335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12335
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
> Fix For: 3.0.x, 3.x
>
> Attachments: 0001-Force-super-column-families-to-be-compound.txt
>
>
> Super Columns are broken after upgrading to cassandra-3.0 HEAD.  The below 
> script shows this.
> 2.1 cli output for get:
> {code}
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> => (name=name, value=Bob, timestamp=1469724504357000)
> {code}
> cqlsh:
> {code}
> [default@test]
>  key  | blobAsText(column1)
> --+-
>  0x53696d6f6e |attr
>  0x426f62 |attr
> {code}
> 3.0 cli:
> {code}
> [default@unknown] use test;
> unconfigured table schema_columnfamilies
> [default@test] get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;
> null
> [default@test]
> {code}
> cqlsh:
> {code}
>  key  | system.blobastext(column1)
> --+--
>  0x53696d6f6e | \x00\x04attr\x00\x00\x04name\x00
>  0x426f62 | \x00\x04attr\x00\x00\x04name\x00
> {code}
> Run this from a directory with cassandra-3.0 checked out and compiled
> {code}
> ccm create -n 2 -v 2.1.14 testsuper
> echo "### Starting 2.1 ###"
> ccm start
> MYFILE=`mktemp`
> echo "create keyspace test with placement_strategy = 
> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = 
> {replication_factor:2};
> use test;
> create column family Sites with column_type = 'Super' and comparator = 
> 'BytesType' and subcomparator='UTF8Type';
> set Sites[utf8('Simon')][utf8('attr')]['name'] = utf8('Simon');
> set Sites[utf8('Bob')][utf8('attr')]['name'] = utf8('Bob');
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7100 flush
> ~/.ccm/repository/2.1.14/bin/nodetool -p 7200 flush
> ccm stop
> # run from cassandra-3.0 checked out and compiled
> ccm setdir
> echo "### Starting Current Directory 
> ###"
> ccm start
> ./bin/nodetool -p 7100 upgradesstables
> ./bin/nodetool -p 7200 upgradesstables
> ./bin/nodetool -p 7100 enablethrift
> ./bin/nodetool -p 7200 enablethrift
> MYFILE=`mktemp`
> echo "use test;
> get Sites[utf8('Bob')][utf8('attr')]['name'] as utf8;" > $MYFILE
> ~/.ccm/repository/2.1.14/bin/cassandra-cli < $MYFILE
> rm $MYFILE
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families

[Sylvain Lebresne (JIRA)]

Sylvain Lebresne created CASSANDRA-12373:


 Summary: 3.0 breaks CQL compatibility with super columns families
 Key: CASSANDRA-12373
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
Reporter: Sylvain Lebresne
 Fix For: 3.0.x, 3.x


This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column 
compatibility.

The details and a proposed solution can be found in the comments of 
CASSANDRA-12335 but the crux of the issue is that super column famillies show 
up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward 
compatibilty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11828) Commit log needs to track unflushed intervals rather than positions

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406047#comment-15406047
 ] 

Sylvain Lebresne commented on CASSANDRA-11828:
--

Thanks, lgtm, I'm +1 on both patches. but could you attach patch and run tests 
for 2.2 (with the 2.1 merged, which seems to have conflicts) and 3.9?

bq. except removing the synchronization from {{IntegerIterval.Set.add}} as the 
individual interval class is thread-safe and hence the consistency is better if 
the set class also

Right. My suggestion to make the base class immutable, with a specifically 
called out mutable (and thread-safe) variant for the case where we need it, as 
I usually like making immutability the default and making it clear when it's 
not, but I'm bikeshedding, especially as long as it's the only use of the 
class. I'm cool with your solution, we can always change things later if 
{IntegerInterval}} sees reuse and immutability is better for those reuse.

> Commit log needs to track unflushed intervals rather than positions
> ---
>
> Key: CASSANDRA-11828
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11828
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> In CASSANDRA-11448 in an effort to give a more thorough handling of flush 
> errors I have introduced a possible correctness bug with disk failure policy 
> ignore if a flush fails with an error:
> - we report the error but continue
> - we correctly do not update the commit log with the flush position
> - but we allow the post-flush executor to resume
> - a successful later flush can thus move the log's clear position beyond the 
> data from the failed flush
> - the log will then delete segment(s) that contain unflushed data.
> After CASSANDRA-9669 it is relatively easy to fix this problem by making the 
> commit log track sets of intervals of unflushed data (as described in 
> CASSANDRA-8496).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11841) Add keep-alive to stream protocol

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11841:
---
Status: Open  (was: Patch Available)

> Add keep-alive to stream protocol
> -
>
> Key: CASSANDRA-11841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11841
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11841) Add keep-alive to stream protocol

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11841:
---
Status: Awaiting Feedback  (was: Open)

> Add keep-alive to stream protocol
> -
>
> Key: CASSANDRA-11841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11841
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12280) nodetool repair hangs

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-12280.
-
Resolution: Cannot Reproduce

Closing since it seems you found a workaround and are no longer able to 
reproduce it. Please reopen if the issue appears again with more detailed DEBUG 
logs and/or reproduction steps.

> nodetool repair hangs
> -
>
> Key: CASSANDRA-12280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12280
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Roth
>
> nodetool repair hangs when repairing a keyspace, does not hang when 
> repairting table/mv by table/mv.
> Command executed (both variants make it hang):
> nodetool repair likes like dislike_by_source_mv like_by_contact_mv 
> match_valid_mv like_out dislike match match_by_contact_mv like_valid_mv 
> like_out_by_source_mv
> OR
> nodetool repair likes
> Logs:
> https://gist.github.com/brstgt/bf8b20fa1942d29ab60926ede7340b75
> Nodetool output:
> https://gist.github.com/brstgt/3aa73662da4b0190630ac1aad6c90a6f
> Schema:
> https://gist.github.com/brstgt/3fd59e0166f86f8065085532e3638097



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12369) dtest failure in thrift_hsha_test.ThriftHSHATest.test_6285

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406010#comment-15406010
 ] 

Joel Knighton commented on CASSANDRA-12369:
---

We presently think this is due to a change in the branch of the Python driver 
we use for the dtests. Those changes have been reverted; if tomorrow's run of 
2.2 without those changes is fixed, I'll close this.

> dtest failure in thrift_hsha_test.ThriftHSHATest.test_6285
> --
>
> Key: CASSANDRA-12369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12369
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: Joel Knighton
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest/664/testReport/thrift_hsha_test/ThriftHSHATest/test_6285



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)