[jira] [Updated] (CASSANDRA-10220) Memtables do not handle column changes for their PartitionColumns

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-10220:
-
Reviewer: Benedict

> Memtables do not handle column changes for their PartitionColumns
> -
>
> Key: CASSANDRA-10220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> There are a couple of inherited issues around the {{PartitionColumns}} in 
> {{AbstractBTreePartition}}. Firstly, we do not always return these columns, 
> but sometimes the metadata columns. Secondly, the columns cannot be updated 
> with new columns, and the metadata columns will not show dropped columns. 
> This should most likely be moved into the {{Holder}}, and updated along with 
> any other updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730664#comment-14730664
 ] 

Robert Stupp commented on CASSANDRA-9590:
-

Oops - lots of dtest failures. [~spo...@gmail.com] can you please take a look 
at the failed dtests (more than 130 for 3.0 and more than 160 for trunk)? It's 
probably always the same issue that manifests in a lot of tests.

> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
> Attachments: nosetest_output.txt
>
>
> Enabling encryption for native transport currently turns SSL exclusively on 
> or off for the opened socket. Migrating from plain to encrypted requires to 
> migrate all native clients as well and redeploy all of them at the same time 
> after starting the SSL enabled Cassandra nodes. 
> This patch would allow to start Cassandra with both an unencrypted and ssl 
> enabled native port. Clients can connect to either, based whether they 
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}} 
> config option. 
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted, 
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set, 
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set, 
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal: 
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
>  [Patch against 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest: 
> [Branch|https://github.com/spodkowinski/cassandra-dtest/tree/cassandra-9590], 
> [Diff 
> master|https://github.com/riptano/cassandra-dtest/compare/master...spodkowinski:cassandra-9590],
>  [Pull Request|https://github.com/riptano/cassandra-dtest/pull/530]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10193) Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to Collection

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730716#comment-14730716
 ] 

Marcus Eriksson commented on CASSANDRA-10193:
-

code looks good, it would be nice with some explicit tests for {{Rows.merge}} 
and {{Rows.diff}} though - I'm sure we have test coverage of them, but it would 
be nice to have direct tests here

> Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to 
> Collection
> 
>
> Key: CASSANDRA-10193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10193
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> There's not really a lot of reason to store a Columns instance in each row. 
> Retaining it introduces extra costs on every row merge, in both consumed CPU 
> time and heap. 
> While working on CASSANDRA-10045 it became apparent this would be very easy 
> to remove, however to avoid scope creep I have filed this as a follow up 
> ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Fix backward compatibility issue due to AbstractBounds serialization bug

[slebresne]

Fix backward compatibility issue due to AbstractBounds serialization bug

patch by slebresne; reviewed by bdeggleston for CASSANDRA-9857


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/24682d21
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/24682d21
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/24682d21

Branch: refs/heads/trunk
Commit: 24682d21d22991deb300ec48527881a532c25c42
Parents: f009272
Author: Sylvain Lebresne 
Authored: Tue Sep 1 17:56:54 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:14:42 2015 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 38 +++-
 .../apache/cassandra/db/ReadResponseTest.java   | 99 
 3 files changed, 133 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/24682d21/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 726eb04..4d8a932 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-beta2
+ * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
  * Fix startup error when upgrading nodes (CASSANDRA-10136)
  * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
  * Improve batchlog write patch (CASSANDRA-9673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/24682d21/src/java/org/apache/cassandra/db/ReadResponse.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadResponse.java 
b/src/java/org/apache/cassandra/db/ReadResponse.java
index 547e7f4..b8ffe25 100644
--- a/src/java/org/apache/cassandra/db/ReadResponse.java
+++ b/src/java/org/apache/cassandra/db/ReadResponse.java
@@ -31,6 +31,7 @@ import org.apache.cassandra.db.filter.ClusteringIndexFilter;
 import org.apache.cassandra.db.filter.ColumnFilter;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.db.partitions.*;
+import org.apache.cassandra.dht.*;
 import org.apache.cassandra.io.IVersionedSerializer;
 import org.apache.cassandra.io.util.DataInputBuffer;
 import org.apache.cassandra.io.util.DataInputPlus;
@@ -221,11 +222,13 @@ public abstract class ReadResponse
  * sorted order, even if the query asks for reversed results.  
Additionally,  pre-3.0 nodes do not have a notion of
  * exclusive slices on non-composite tables, so extra rows may need to be 
trimmed.
  */
-private static class LegacyRemoteDataResponse extends ReadResponse
+@VisibleForTesting
+static class LegacyRemoteDataResponse extends ReadResponse
 {
 private final List partitions;
 
-private LegacyRemoteDataResponse(List 
partitions)
+@VisibleForTesting
+LegacyRemoteDataResponse(List partitions)
 {
 super(null); // we never serialize LegacyRemoteDataResponses, so 
we don't care about the metadata
 this.partitions = partitions;
@@ -233,6 +236,31 @@ public abstract class ReadResponse
 
 public UnfilteredPartitionIterator makeIterator(CFMetaData metadata, 
final ReadCommand command)
 {
+// Due to a bug in the serialization of AbstractBounds, anything 
that isn't a Range is understood by pre-3.0 nodes
+// as a Bound, which means IncludingExcludingBounds and 
ExcludingBounds responses may include keys they shouldn't.
+// So filter partitions that shouldn't be included here.
+boolean skipFirst = false;
+boolean skipLast = false;
+if (!partitions.isEmpty() && command instanceof 
PartitionRangeReadCommand)
+{
+AbstractBounds keyRange = 
((PartitionRangeReadCommand)command).dataRange().keyRange();
+boolean isExcludingBounds = keyRange instanceof 
ExcludingBounds;
+skipFirst = isExcludingBounds && 
!keyRange.contains(partitions.get(0).partitionKey());
+skipLast = (isExcludingBounds || keyRange instanceof 
IncludingExcludingBounds) && 
!keyRange.contains(partitions.get(partitions.size() - 1).partitionKey());
+}
+
+final List toReturn;
+if (skipFirst || skipLast)
+{
+toReturn = partitions.size() == 1
+ ? Collections.emptyList()
+ : partitions.subList(skipFirst ? 1 : 0, skipLast ? 
partitions.size() - 1 : partitions.size());
+}
+else
+{
+toReturn = partitions;
+}
+
 return new AbstractUnfilteredPartitionIterator()
 {
 private 

cassandra git commit: Fix backward compatibility issue due to AbstractBounds serialization bug

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 f009272ba -> 24682d21d


Fix backward compatibility issue due to AbstractBounds serialization bug

patch by slebresne; reviewed by bdeggleston for CASSANDRA-9857


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/24682d21
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/24682d21
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/24682d21

Branch: refs/heads/cassandra-3.0
Commit: 24682d21d22991deb300ec48527881a532c25c42
Parents: f009272
Author: Sylvain Lebresne 
Authored: Tue Sep 1 17:56:54 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:14:42 2015 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 38 +++-
 .../apache/cassandra/db/ReadResponseTest.java   | 99 
 3 files changed, 133 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/24682d21/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 726eb04..4d8a932 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-beta2
+ * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
  * Fix startup error when upgrading nodes (CASSANDRA-10136)
  * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
  * Improve batchlog write patch (CASSANDRA-9673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/24682d21/src/java/org/apache/cassandra/db/ReadResponse.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadResponse.java 
b/src/java/org/apache/cassandra/db/ReadResponse.java
index 547e7f4..b8ffe25 100644
--- a/src/java/org/apache/cassandra/db/ReadResponse.java
+++ b/src/java/org/apache/cassandra/db/ReadResponse.java
@@ -31,6 +31,7 @@ import org.apache.cassandra.db.filter.ClusteringIndexFilter;
 import org.apache.cassandra.db.filter.ColumnFilter;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.db.partitions.*;
+import org.apache.cassandra.dht.*;
 import org.apache.cassandra.io.IVersionedSerializer;
 import org.apache.cassandra.io.util.DataInputBuffer;
 import org.apache.cassandra.io.util.DataInputPlus;
@@ -221,11 +222,13 @@ public abstract class ReadResponse
  * sorted order, even if the query asks for reversed results.  
Additionally,  pre-3.0 nodes do not have a notion of
  * exclusive slices on non-composite tables, so extra rows may need to be 
trimmed.
  */
-private static class LegacyRemoteDataResponse extends ReadResponse
+@VisibleForTesting
+static class LegacyRemoteDataResponse extends ReadResponse
 {
 private final List partitions;
 
-private LegacyRemoteDataResponse(List 
partitions)
+@VisibleForTesting
+LegacyRemoteDataResponse(List partitions)
 {
 super(null); // we never serialize LegacyRemoteDataResponses, so 
we don't care about the metadata
 this.partitions = partitions;
@@ -233,6 +236,31 @@ public abstract class ReadResponse
 
 public UnfilteredPartitionIterator makeIterator(CFMetaData metadata, 
final ReadCommand command)
 {
+// Due to a bug in the serialization of AbstractBounds, anything 
that isn't a Range is understood by pre-3.0 nodes
+// as a Bound, which means IncludingExcludingBounds and 
ExcludingBounds responses may include keys they shouldn't.
+// So filter partitions that shouldn't be included here.
+boolean skipFirst = false;
+boolean skipLast = false;
+if (!partitions.isEmpty() && command instanceof 
PartitionRangeReadCommand)
+{
+AbstractBounds keyRange = 
((PartitionRangeReadCommand)command).dataRange().keyRange();
+boolean isExcludingBounds = keyRange instanceof 
ExcludingBounds;
+skipFirst = isExcludingBounds && 
!keyRange.contains(partitions.get(0).partitionKey());
+skipLast = (isExcludingBounds || keyRange instanceof 
IncludingExcludingBounds) && 
!keyRange.contains(partitions.get(partitions.size() - 1).partitionKey());
+}
+
+final List toReturn;
+if (skipFirst || skipLast)
+{
+toReturn = partitions.size() == 1
+ ? Collections.emptyList()
+ : partitions.subList(skipFirst ? 1 : 0, skipLast ? 
partitions.size() - 1 : partitions.size());
+}
+else
+{
+toReturn = partitions;
+}
+
   

[jira] [Issue Comment Deleted] (CASSANDRA-10255) AttributeError: 'module' object has no attribute 'compress'

[navithejesh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

navithejesh updated CASSANDRA-10255:

Comment: was deleted

(was: This is the nodetool status of two nodes)

> AttributeError: 'module' object has no attribute 'compress' 
> 
>
> Key: CASSANDRA-10255
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10255
> Project: Cassandra
>  Issue Type: Bug
>  Components: Config, Packaging
> Environment: Linux fedora 21
>Reporter: navithejesh
>  Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: erroe2.jpeg, error.jpeg
>
>
> Hi Recently i have upgrade my server from cassandra version 2.0.11 to 2.1.8 
> using tarball installation.And i have added one node to existing culster with 
> the same version of 2.1.8. All the configuration settings are done but am not 
> able to run cqlsh from bin directory.and this is the error
> AttributeError: 'module' object has no attribute 'compress'
> {code}
> ./cqlsh
> Traceback (most recent call last):
>  File "./cqlsh", line 111, in 
> from cassandra.cluster import Cluster, PagedResult
> File 
> "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 49, in 
>  File 
> "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/connection.py",
>  line 88, in 
> AttributeError: 'module' object has no attribute 'compress'
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-10255) AttributeError: 'module' object has no attribute 'compress'

[navithejesh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

navithejesh updated CASSANDRA-10255:

Comment: was deleted

(was: This is the nodetool status of two nodes A and B.)

> AttributeError: 'module' object has no attribute 'compress' 
> 
>
> Key: CASSANDRA-10255
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10255
> Project: Cassandra
>  Issue Type: Bug
>  Components: Config, Packaging
> Environment: Linux fedora 21
>Reporter: navithejesh
>  Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: erroe2.jpeg, error.jpeg
>
>
> Hi Recently i have upgrade my server from cassandra version 2.0.11 to 2.1.8 
> using tarball installation.And i have added one node to existing culster with 
> the same version of 2.1.8. All the configuration settings are done but am not 
> able to run cqlsh from bin directory.and this is the error
> AttributeError: 'module' object has no attribute 'compress'
> {code}
> ./cqlsh
> Traceback (most recent call last):
>  File "./cqlsh", line 111, in 
> from cassandra.cluster import Cluster, PagedResult
> File 
> "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 49, in 
>  File 
> "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/connection.py",
>  line 88, in 
> AttributeError: 'module' object has no attribute 'compress'
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7464) Retire/replace sstable2json and json2sstable

[Tupshin Harper (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730687#comment-14730687
 ] 

Tupshin Harper commented on CASSANDRA-7464:
---

With sstable2json going away with 3.0, but no activity nor timeline for this 
ticket, it seems like we are going to be left in a situation where we have no 
way to debug the contents of an sstable. This would seem to be a requirement 
for a 3.0-final release.

> Retire/replace sstable2json and json2sstable
> 
>
> Key: CASSANDRA-7464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7464
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Priority: Minor
>
> Both tools are pretty awful. They are primarily meant for debugging (there is 
> much more efficient and convenient ways to do import/export data), but their 
> output manage to be hard to handle both for humans and for tools (especially 
> as soon as you have modern stuff like composites).
> There is value to having tools to export sstable contents into a format that 
> is easy to manipulate by human and tools for debugging, small hacks and 
> general tinkering, but sstable2json and json2sstable are not that.  
> So I propose that we deprecate those tools and consider writing better 
> replacements. It shouldn't be too hard to come up with an output format that 
> is more aware of modern concepts like composites, UDTs, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730685#comment-14730685
 ] 

Benedict commented on CASSANDRA-8630:
-

Thanks. Committed as ce63ccc842dc6e7129765391c611402eb02a3a23.

> Faster sequential IO (on compaction, streaming, etc)
> 
>
> Key: CASSANDRA-8630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8630
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core, Tools
>Reporter: Oleg Anastasyev
>Assignee: Stefania
>  Labels: compaction, performance
> Fix For: 3.x
>
> Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, 
> flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, 
> mmaped_uncomp_hotspot.png
>
>
> When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot 
> of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int).
> This is because default implementations of readShort,readLong, etc as well as 
> their matching write* are implemented with numerous calls of byte by byte 
> read and write. 
> This makes a lot of syscalls as well.
> A quick microbench shows than just reimplementation of these methods in 
> either way gives 8x speed increase.
> A patch attached implements RandomAccessReader.read and 
> SequencialWriter.write methods in more efficient way.
> I also eliminated some extra byte copies in CompositeType.split and 
> ColumnNameHelper.maxComponents, which were on my profiler's hotspot method 
> list during tests.
> A stress tests on my laptop show that this patch makes compaction 25-30% 
> faster  on uncompressed sstables and 15% faster for compressed ones.
> A deployment to production shows much less CPU load for compaction. 
> (I attached a cpu load graph from one of our production, orange is niced CPU 
> load - i.e. compaction; yellow is user - i.e. not compaction related tasks)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10193) Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to Collection

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730718#comment-14730718
 ] 

Benedict commented on CASSANDRA-10193:
--

Agreed. Filed CASSANDRA-10266, so as not to delay this past beta2.

> Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to 
> Collection
> 
>
> Key: CASSANDRA-10193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10193
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> There's not really a lot of reason to store a Columns instance in each row. 
> Retaining it introduces extra costs on every row merge, in both consumed CPU 
> time and heap. 
> While working on CASSANDRA-10045 it became apparent this would be very easy 
> to remove, however to avoid scope creep I have filed this as a follow up 
> ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10220) Memtables do not handle column changes for their PartitionColumns

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730653#comment-14730653
 ] 

Benedict commented on CASSANDRA-10220:
--

LGTM, so +1 assuming CI passes.

> Memtables do not handle column changes for their PartitionColumns
> -
>
> Key: CASSANDRA-10220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> There are a couple of inherited issues around the {{PartitionColumns}} in 
> {{AbstractBTreePartition}}. Firstly, we do not always return these columns, 
> but sometimes the metadata columns. Secondly, the columns cannot be updated 
> with new columns, and the metadata columns will not show dropped columns. 
> This should most likely be moved into the {{Holder}}, and updated along with 
> any other updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10220) Memtables do not handle column changes for their PartitionColumns

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-10220:
-
Assignee: Sylvain Lebresne  (was: Benedict)

> Memtables do not handle column changes for their PartitionColumns
> -
>
> Key: CASSANDRA-10220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> There are a couple of inherited issues around the {{PartitionColumns}} in 
> {{AbstractBTreePartition}}. Firstly, we do not always return these columns, 
> but sometimes the metadata columns. Secondly, the columns cannot be updated 
> with new columns, and the metadata columns will not show dropped columns. 
> This should most likely be moved into the {{Holder}}, and updated along with 
> any other updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[08/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/src/java/org/apache/cassandra/io/util/AbstractDataInput.java
--
diff --git a/src/java/org/apache/cassandra/io/util/AbstractDataInput.java 
b/src/java/org/apache/cassandra/io/util/AbstractDataInput.java
deleted file mode 100644
index 29ce2c3..000
--- a/src/java/org/apache/cassandra/io/util/AbstractDataInput.java
+++ /dev/null
@@ -1,343 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.io.util;
-
-import java.io.*;
-
-public abstract class AbstractDataInput extends InputStream implements 
DataInputPlus
-{
-public abstract void seek(long position) throws IOException;
-public abstract long getPosition();
-public abstract long getPositionLimit();
-
-public int skipBytes(int n) throws IOException
-{
-if (n <= 0)
-return 0;
-long oldPosition = getPosition();
-seek(Math.min(getPositionLimit(), oldPosition + n));
-long skipped = getPosition() - oldPosition;
-assert skipped >= 0 && skipped <= n;
-return (int) skipped;
-}
-
-/**
- * Reads a boolean from the current position in this file. Blocks until one
- * byte has been read, the end of the file is reached or an exception is
- * thrown.
- *
- * @return the next boolean value from this file.
- * @throws java.io.EOFException
- * if the end of this file is detected.
- * @throws java.io.IOException
- * if this file is closed or another I/O error occurs.
- */
-public final boolean readBoolean() throws IOException {
-int temp = this.read();
-if (temp < 0) {
-throw new EOFException();
-}
-return temp != 0;
-}
-
-/**
- * Reads an 8-bit byte from the current position in this file. Blocks until
- * one byte has been read, the end of the file is reached or an exception 
is
- * thrown.
- *
- * @return the next signed 8-bit byte value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-public final byte readByte() throws IOException {
-int temp = this.read();
-if (temp < 0) {
-throw new EOFException();
-}
-return (byte) temp;
-}
-
-/**
- * Reads a 16-bit character from the current position in this file. Blocks 
until
- * two bytes have been read, the end of the file is reached or an 
exception is
- * thrown.
- *
- * @return the next char value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-public final char readChar() throws IOException {
-int ch1 = this.read();
-int ch2 = this.read();
-if ((ch1 | ch2) < 0)
-throw new EOFException();
-return (char)((ch1 << 8) + (ch2 << 0));
-}
-
-/**
- * Reads a 64-bit double from the current position in this file. Blocks
- * until eight bytes have been read, the end of the file is reached or an
- * exception is thrown.
- *
- * @return the next double value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-public final double readDouble() throws IOException {
-return Double.longBitsToDouble(readLong());
-}
-
-/**
- * Reads a 32-bit float from the current position in this file. Blocks
- * until four bytes have been read, the end of the file is reached or an
- * exception is thrown.
- *
- * @return the next float value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-  

[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[benedict]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/356dd5b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/356dd5b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/356dd5b3

Branch: refs/heads/trunk
Commit: 356dd5b31f73628ade6ea28efe33c55d1d1d3379
Parents: eaf0e35 ce63ccc
Author: Benedict Elliott Smith 
Authored: Fri Sep 4 12:43:37 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 12:43:37 2015 +0100

--
 build.xml   |   8 +-
 lib/licenses/ohc-0.3.4.txt  | 201 
 lib/licenses/ohc-0.4.2.txt  | 201 
 lib/ohc-core-0.4.2.jar  | Bin 0 -> 126802 bytes
 lib/ohc-core-0.4.jar| Bin 127890 -> 0 bytes
 lib/ohc-core-j8-0.4.2.jar   | Bin 0 -> 4994 bytes
 lib/ohc-core-j8-0.4.jar | Bin 4989 -> 0 bytes
 .../apache/cassandra/cache/AutoSavingCache.java |   2 +-
 .../org/apache/cassandra/cache/OHCProvider.java |   3 +-
 .../org/apache/cassandra/db/SystemKeyspace.java |   2 +-
 .../db/commitlog/CommitLogReplayer.java |  19 +-
 .../cassandra/db/filter/ColumnFilter.java   |   2 +-
 .../cassandra/hints/ChecksummedDataInput.java   | 128 +++--
 .../org/apache/cassandra/hints/HintsReader.java |  67 ++-
 .../compress/CompressedRandomAccessReader.java  | 147 +++---
 .../io/compress/CompressedThrottledReader.java  |  48 --
 .../io/compress/CompressionMetadata.java|   2 +-
 .../cassandra/io/sstable/KeyIterator.java   |   1 -
 .../io/sstable/format/SSTableReader.java| 100 ++--
 .../cassandra/io/sstable/format/Version.java|   2 +
 .../io/sstable/format/big/BigFormat.java|   9 +
 .../io/sstable/format/big/BigTableReader.java   | 108 ++---
 .../io/sstable/format/big/BigTableWriter.java   |   2 -
 .../cassandra/io/util/AbstractDataInput.java| 343 -
 .../io/util/BufferedSegmentedFile.java  |  24 -
 .../cassandra/io/util/ByteBufferDataInput.java  | 171 ---
 .../apache/cassandra/io/util/ChannelProxy.java  |   4 +-
 .../io/util/ChecksummedRandomAccessReader.java  |  62 ++-
 .../io/util/CompressedSegmentedFile.java| 100 ++--
 .../cassandra/io/util/DataInputBuffer.java  |  24 +-
 .../io/util/DataIntegrityMetadata.java  |  11 +-
 .../apache/cassandra/io/util/FileDataInput.java |  25 +-
 .../io/util/FileSegmentInputStream.java |  96 
 .../cassandra/io/util/ICompressedFile.java  |   9 +-
 .../cassandra/io/util/MemoryInputStream.java|  54 ++-
 .../cassandra/io/util/MmappedRegions.java   | 344 +
 .../cassandra/io/util/MmappedSegmentedFile.java | 214 +++-
 .../cassandra/io/util/NIODataInputStream.java   | 352 +-
 .../cassandra/io/util/RandomAccessReader.java   | 417 ++--
 .../io/util/RebufferingInputStream.java | 286 +++
 .../apache/cassandra/io/util/SegmentedFile.java | 115 ++---
 .../cassandra/io/util/ThrottledReader.java  |  48 --
 .../compress/CompressedStreamWriter.java|  10 +-
 .../apache/cassandra/utils/ByteBufferUtil.java  |   3 -
 .../org/apache/cassandra/utils/Throwables.java  |  45 +-
 .../apache/cassandra/utils/vint/VIntCoding.java |   2 +-
 .../cassandra/db/commitlog/CommitLogTest.java   |   8 +-
 .../db/commitlog/CommitLogTestReplayer.java |   3 +-
 .../hints/ChecksummedDataInputTest.java | 227 +++--
 .../io/ChecksummedRandomAccessReaderTest.java   | 127 -
 .../cassandra/io/RandomAccessReaderTest.java| 269 ---
 .../CompressedRandomAccessReaderTest.java   | 245 +-
 .../CompressedSequentialWriterTest.java |   2 +-
 .../cassandra/io/sstable/SSTableReaderTest.java |   3 +-
 .../io/util/BufferedDataOutputStreamTest.java   |  12 +-
 .../io/util/BufferedRandomAccessFileTest.java   |  91 +---
 .../util/ChecksummedRandomAccessReaderTest.java | 127 +
 .../io/util/FileSegmentInputStreamTest.java | 131 +
 .../apache/cassandra/io/util/MemoryTest.java|  37 ++
 .../cassandra/io/util/MmappedRegionsTest.java   | 375 ++
 .../io/util/NIODataInputStreamTest.java |  16 +-
 .../io/util/RandomAccessReaderTest.java | 483 +++
 62 files changed, 3340 insertions(+), 2627 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/356dd5b3/src/java/org/apache/cassandra/io/util/DataIntegrityMetadata.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/356dd5b3/src/java/org/apache/cassandra/utils/ByteBufferUtil.java

[06/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
index ca15722..1bdc591 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
@@ -117,7 +117,7 @@ public class CompressedSequentialWriterTest extends 
SequentialWriterTest
 }
 
 assert f.exists();
-RandomAccessReader reader = 
CompressedRandomAccessReader.open(channel, new CompressionMetadata(filename + 
".metadata", f.length(), ChecksumType.CRC32));
+RandomAccessReader reader = new 
CompressedRandomAccessReader.Builder(channel, new CompressionMetadata(filename 
+ ".metadata", f.length(), ChecksumType.CRC32)).build();
 assertEquals(dataPre.length + rawPost.length, reader.length());
 byte[] result = new byte[(int)reader.length()];
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java 
b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
index 793348a..c7f3c36 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
@@ -46,6 +46,7 @@ import org.apache.cassandra.dht.Token;
 import org.apache.cassandra.index.Index;
 import org.apache.cassandra.io.sstable.format.SSTableReader;
 import org.apache.cassandra.io.util.FileDataInput;
+import org.apache.cassandra.io.util.MmappedRegions;
 import org.apache.cassandra.io.util.MmappedSegmentedFile;
 import org.apache.cassandra.io.util.SegmentedFile;
 import org.apache.cassandra.schema.CachingParams;
@@ -133,7 +134,7 @@ public class SSTableReaderTest
 @Test
 public void testSpannedIndexPositions() throws IOException
 {
-MmappedSegmentedFile.MAX_SEGMENT_SIZE = 40; // each index entry is ~11 
bytes, so this will generate lots of segments
+MmappedRegions.MAX_SEGMENT_SIZE = 40; // each index entry is ~11 
bytes, so this will generate lots of segments
 
 Keyspace keyspace = Keyspace.open(KEYSPACE1);
 ColumnFamilyStore store = keyspace.getColumnFamilyStore("Standard1");

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java 
b/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
index b875a6a..4a72281 100644
--- a/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
+++ b/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
@@ -542,14 +542,12 @@ public class BufferedDataOutputStreamTest
 
 ndosp.flush();
 
-@SuppressWarnings("resource")
-ByteBufferDataInput bbdi = new 
ByteBufferDataInput(ByteBuffer.wrap(generated.toByteArray()), "", 0, 0);
-
+DataInputBuffer in = new DataInputBuffer(generated.toByteArray());
 assertEquals(expectedSize, generated.toByteArray().length);
 
 for (long v : testValues)
 {
-assertEquals(v, bbdi.readVInt());
+assertEquals(v, in.readVInt());
 }
 }
 
@@ -574,13 +572,11 @@ public class BufferedDataOutputStreamTest
 
 ndosp.flush();
 
-@SuppressWarnings("resource")
-ByteBufferDataInput bbdi = new 
ByteBufferDataInput(ByteBuffer.wrap(generated.toByteArray()), "", 0, 0);
-
+DataInputBuffer in = new DataInputBuffer(generated.toByteArray());
 assertEquals(expectedSize, generated.toByteArray().length);
 
 for (long v : testValues)
-assertEquals(v, bbdi.readUnsignedVInt());
+assertEquals(v, in.readUnsignedVInt());
 }
 
 @Test

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java 
b/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
index e051c00..364ea71 100644
--- a/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
+++ b/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
@@ -25,9 +25,7 @@ import java.io.File;
 import java.io.FileOutputStream;
 import 

[01/10] cassandra git commit: Slightly better ColumnFilter.toString()

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 7c0bfe992 -> ce63ccc84
  refs/heads/trunk eaf0e3523 -> 356dd5b31


Slightly better ColumnFilter.toString()


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c0bfe99
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c0bfe99
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c0bfe99

Branch: refs/heads/trunk
Commit: 7c0bfe99223566af961fcb4f426ba8c3342ed473
Parents: 24682d2
Author: Sylvain Lebresne 
Authored: Fri Sep 4 11:33:43 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:33:43 2015 +0200

--
 src/java/org/apache/cassandra/db/filter/ColumnFilter.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c0bfe99/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java 
b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
index 2afc785..9205ff9 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
@@ -306,7 +306,7 @@ public class ColumnFilter
 @Override
 public String toString()
 {
-if (selection == null)
+if (isFetchAll)
 return "*";
 
 if (selection.isEmpty())

[03/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
--
diff --git a/src/java/org/apache/cassandra/io/util/RandomAccessReader.java 
b/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
index b13d154..e9b0ee4 100644
--- a/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
+++ b/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
@@ -19,119 +19,129 @@ package org.apache.cassandra.io.util;
 
 import java.io.*;
 import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+
+import com.google.common.primitives.Ints;
+import com.google.common.util.concurrent.RateLimiter;
 
 import org.apache.cassandra.io.FSReadError;
 import org.apache.cassandra.io.compress.BufferType;
-import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.memory.BufferPool;
 
-public class RandomAccessReader extends AbstractDataInput implements 
FileDataInput
+public class RandomAccessReader extends RebufferingInputStream implements 
FileDataInput
 {
+// The default buffer size when the client doesn't specify it
 public static final int DEFAULT_BUFFER_SIZE = 4096;
 
+// The maximum buffer size when the limiter is not null, i.e. when 
throttling
+// is enabled. This is required to avoid aquiring permits that are too 
large.
+public static final int MAX_THROTTLED_BUFFER_SIZE = 1 << 16; // 64k
+
 // the IO channel to the file, we do not own a reference to this due to
 // performance reasons (CASSANDRA-9379) so it's up to the owner of the RAR 
to
 // ensure that the channel stays open and that it is closed afterwards
 protected final ChannelProxy channel;
 
-// buffer which will cache file blocks
-protected ByteBuffer buffer;
+// optional memory mapped regions for the channel
+protected final MmappedRegions regions;
 
-// `bufferOffset` is the offset of the beginning of the buffer
-// `markedPointer` folds the offset of the last file mark
-protected long bufferOffset, markedPointer;
+// An optional limiter that will throttle the amount of data we read
+protected final RateLimiter limiter;
 
-// this can be overridden at construction to a value shorter than the true 
length of the file;
-// if so, it acts as an imposed limit on reads, rather than a convenience 
property
+// the file length, this can be overridden at construction to a value 
shorter
+// than the true length of the file; if so, it acts as an imposed limit on 
reads,
+// required when opening sstables early not to read past the mark
 private final long fileLength;
 
-protected RandomAccessReader(ChannelProxy channel, int bufferSize, long 
overrideLength, BufferType bufferType)
-{
-this.channel = channel;
+// the buffer size for buffered readers
+protected final int bufferSize;
 
-if (bufferSize <= 0)
-throw new IllegalArgumentException("bufferSize must be positive");
+// the buffer type for buffered readers
+private final BufferType bufferType;
 
-// we can cache file length in read-only mode
-fileLength = overrideLength <= 0 ? channel.size() : overrideLength;
+// offset from the beginning of the file
+protected long bufferOffset;
 
-buffer = allocateBuffer(getBufferSize(bufferSize), bufferType);
-buffer.limit(0);
-}
+// offset of the last file mark
+protected long markedPointer;
 
-/** The buffer size is typically already page aligned but if that is not 
the case
- * make sure that it is a multiple of the page size, 4096.
- * */
-protected int getBufferSize(int size)
+protected RandomAccessReader(Builder builder)
 {
-if ((size & ~4095) != size)
-{ // should already be a page size multiple but if that's not case 
round it up
-size = (size + 4095) & ~4095;
-}
-return size;
-}
+super(null);
 
-protected ByteBuffer allocateBuffer(int size, BufferType bufferType)
-{
-return BufferPool.get(size, bufferType);
-}
+this.channel = builder.channel;
+this.regions = builder.regions;
+this.limiter = builder.limiter;
+this.fileLength = builder.overrideLength <= 0 ? builder.channel.size() 
: builder.overrideLength;
+this.bufferSize = getBufferSize(builder);
+this.bufferType = builder.bufferType;
 
-// A wrapper of the RandomAccessReader that closes the channel when done.
-// For performance reasons RAR does not increase the reference count of
-// a channel but assumes the owner will keep it open and close it,
-// see CASSANDRA-9379, this thin class is just for those cases where we do
-// not have a shared channel.
-private static class RandomAccessReaderWithChannel extends 
RandomAccessReader
-{
-@SuppressWarnings("resource")
- 

[jira] [Created] (CASSANDRA-10266) Introduce direct unit test coverage for Rows

[Benedict (JIRA)]

Benedict created CASSANDRA-10266:


 Summary: Introduce direct unit test coverage for Rows
 Key: CASSANDRA-10266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10266
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.0.0 rc1


As with much of the codebase, we have no direct unit test coverage for 
{{Rows}}, and we should remedy this given how central it is to behaviour.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10109) Windows dtest 3.0: ttl_test.py failures

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730544#comment-14730544
 ] 

Stefania commented on CASSANDRA-10109:
--

CI is running, including utests on win:

http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10109-3.0-dtest/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10109-3.0-testall/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10109_utest_win32/

> Windows dtest 3.0: ttl_test.py failures
> ---
>
> Key: CASSANDRA-10109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10109
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Stefania
>  Labels: Windows
> Fix For: 3.0.0 rc1
>
>
> ttl_test.py:TestTTL.update_column_ttl_with_default_ttl_test2
> ttl_test.py:TestTTL.update_multiple_columns_ttl_test
> ttl_test.py:TestTTL.update_single_column_ttl_test
> Errors locally are different than CI from yesterday. Yesterday on CI we have 
> timeouts and general node hangs. Today on all 3 tests when run locally I see:
> {noformat}
> Traceback (most recent call last):
>   File "c:\src\cassandra-dtest\dtest.py", line 532, in tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name, 
> errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [main] 2015-08-17 
> 16:53:43,120 NoSpamLogger.java:97 - This platform does not support atomic 
> directory streams (SecureDirectoryStream); race conditions when loading 
> sstable files could occurr']
> {noformat}
> This traces back to the commit for CASSANDRA-7066 today by [~Stefania] and 
> [~benedict].  Stefania - care to take this ticket and also look further into 
> whether or not we're going to have issues with 7066 on Windows? That error 
> message certainly *sounds* like it's not a good thing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix startup error when upgrading nodes to 3.0

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 dc5b268d0 -> f009272ba


Fix startup error when upgrading nodes to 3.0

patch by slebresne; reviewed by blambov for CASSANDRA-10136


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f009272b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f009272b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f009272b

Branch: refs/heads/cassandra-3.0
Commit: f009272ba6ffb94579a443733f5cfe25ec9606fa
Parents: dc5b268
Author: Sylvain Lebresne 
Authored: Wed Aug 19 16:13:52 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:11:42 2015 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/Serializers.java| 65 ++--
 .../db/SinglePartitionNamesCommand.java |  2 +-
 .../columniterator/AbstractSSTableIterator.java | 36 +--
 .../db/columniterator/SSTableIterator.java  |  4 +-
 .../columniterator/SSTableReversedIterator.java | 35 +--
 .../cassandra/db/marshal/CompositeType.java |  5 ++
 .../cassandra/io/sstable/IndexHelper.java   |  6 +-
 8 files changed, 133 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f009272b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fb33de0..726eb04 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-beta2
+ * Fix startup error when upgrading nodes (CASSANDRA-10136)
  * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
  * Improve batchlog write patch (CASSANDRA-9673)
  * Re-apply MaterializedView updates on commitlog replay (CASSANDRA-10164)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f009272b/src/java/org/apache/cassandra/db/Serializers.java
--
diff --git a/src/java/org/apache/cassandra/db/Serializers.java 
b/src/java/org/apache/cassandra/db/Serializers.java
index 8056222..2561bbe 100644
--- a/src/java/org/apache/cassandra/db/Serializers.java
+++ b/src/java/org/apache/cassandra/db/Serializers.java
@@ -18,12 +18,16 @@
 package org.apache.cassandra.db;
 
 import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.List;
 
 import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.db.marshal.CompositeType;
 import org.apache.cassandra.io.ISerializer;
 import org.apache.cassandra.io.util.DataInputPlus;
 import org.apache.cassandra.io.util.DataOutputPlus;
 import org.apache.cassandra.io.sstable.format.Version;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 import static org.apache.cassandra.io.sstable.IndexHelper.IndexInfo;
 
@@ -44,12 +48,65 @@ public class Serializers
 return new IndexInfo.Serializer(metadata, version);
 }
 
-// Note that for the old layout, this will actually discard the cellname 
parts that are not strictly
-// part of the clustering prefix. Don't use this if that's not what you 
want.
-public ISerializer clusteringPrefixSerializer(final 
Version version, final SerializationHeader header)
+// TODO: Once we drop support for old (pre-3.0) sstables, we can drop this 
method and inline the calls to
+// ClusteringPrefix.serializer in IndexHelper directly. At which point 
this whole class probably becomes
+// unecessary (since IndexInfo.Serializer won't depend on the metadata 
either).
+public ISerializer 
indexEntryClusteringPrefixSerializer(final Version version, final 
SerializationHeader header)
 {
 if (!version.storeRows())
-throw new UnsupportedOperationException();
+{
+return new ISerializer()
+{
+public void serialize(ClusteringPrefix clustering, 
DataOutputPlus out) throws IOException
+{
+// We should only use this for reading old sstable, never 
write new ones.
+throw new UnsupportedOperationException();
+}
+
+public ClusteringPrefix deserialize(DataInputPlus in) throws 
IOException
+{
+// We're reading the old cellname/composite
+ByteBuffer bb = ByteBufferUtil.readWithShortLength(in);
+assert bb.hasRemaining(); // empty cellnames were invalid
+
+int clusteringSize = metadata.clusteringColumns().size();
+// If the table has no clustering column, then the 
cellname will just be the "column" name, which we ignore here.
+if (clusteringSize == 0)
+return Clustering.EMPTY;
+
+if 

[jira] [Commented] (CASSANDRA-10193) Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to Collection

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730587#comment-14730587
 ] 

Benedict commented on CASSANDRA-10193:
--

Thanks. Rebased and force-pushed to same repository.

> Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to 
> Collection
> 
>
> Key: CASSANDRA-10193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10193
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> There's not really a lot of reason to store a Columns instance in each row. 
> Retaining it introduces extra costs on every row merge, in both consumed CPU 
> time and heap. 
> While working on CASSANDRA-10045 it became apparent this would be very easy 
> to remove, however to avoid scope creep I have filed this as a follow up 
> ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7950) Output of nodetool compactionstats and compactionhistory does not work well with long keyspace and column family names.

[Eugene (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730579#comment-14730579
 ] 

Eugene commented on CASSANDRA-7950:
---

Thank you to everyone that has worked on this so far.

Will this, by chance, also fix the output of other commands, for example 
`nodetool listsnapshots`?

I created a snapshot with a long tag 
(some-really-long-snapshot-name-geezey-petes). Ended up with the following:

{code}
# nodetool listsnapshots
Snapshot Details: 
Snapshot name   Keyspace nameColumn family name   
True size  Size on disk   
some-really-long-snapshot-name-geezey-petesgalactaron   
replicatorstide  0 bytes570.68 MB  
some-really-long-snapshot-name-geezey-petesdse_system   leases  
 0 bytes13 bytes   
some-really-long-snapshot-name-geezey-petesdse_system   
real_leaders 0 bytes13 bytes   
some-really-long-snapshot-name-geezey-petesdse_system   
encrypted_keys   0 bytes13 bytes   
some-really-long-snapshot-name-geezey-petesdse_system   
registered_leaders   0 bytes13 bytes   
{code}

> Output of nodetool compactionstats and compactionhistory does not work well 
> with long keyspace and column family names.  
> -
>
> Key: CASSANDRA-7950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7950
> Project: Cassandra
>  Issue Type: Bug
> Environment: CentOS 5, 64bit, Oracle JDK 7, DSE
>Reporter: Eugene
>Assignee: Michael Shuler
>Priority: Minor
>  Labels: lhf
> Fix For: 2.1.x
>
> Attachments: 7950.patch, nodetool-examples.txt
>
>
> When running these commands:
> nodetool compactionstats
> nodetool compactionhistory
> The output can be difficult to grok due to long keyspace names, column family 
> names, and long values.  I have attached an example.
> It's difficult for both humans and grep/sed/awk/perl to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Slightly better ColumnFilter.toString()

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 24682d21d -> 7c0bfe992


Slightly better ColumnFilter.toString()


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c0bfe99
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c0bfe99
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c0bfe99

Branch: refs/heads/cassandra-3.0
Commit: 7c0bfe99223566af961fcb4f426ba8c3342ed473
Parents: 24682d2
Author: Sylvain Lebresne 
Authored: Fri Sep 4 11:33:43 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:33:43 2015 +0200

--
 src/java/org/apache/cassandra/db/filter/ColumnFilter.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c0bfe99/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java 
b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
index 2afc785..9205ff9 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
@@ -306,7 +306,7 @@ public class ColumnFilter
 @Override
 public String toString()
 {
-if (selection == null)
+if (isFetchAll)
 return "*";
 
 if (selection.isEmpty())

[02/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
 
b/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
index ca15722..1bdc591 100644
--- 
a/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
+++ 
b/test/unit/org/apache/cassandra/io/compress/CompressedSequentialWriterTest.java
@@ -117,7 +117,7 @@ public class CompressedSequentialWriterTest extends 
SequentialWriterTest
 }
 
 assert f.exists();
-RandomAccessReader reader = 
CompressedRandomAccessReader.open(channel, new CompressionMetadata(filename + 
".metadata", f.length(), ChecksumType.CRC32));
+RandomAccessReader reader = new 
CompressedRandomAccessReader.Builder(channel, new CompressionMetadata(filename 
+ ".metadata", f.length(), ChecksumType.CRC32)).build();
 assertEquals(dataPre.length + rawPost.length, reader.length());
 byte[] result = new byte[(int)reader.length()];
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java 
b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
index 793348a..c7f3c36 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
@@ -46,6 +46,7 @@ import org.apache.cassandra.dht.Token;
 import org.apache.cassandra.index.Index;
 import org.apache.cassandra.io.sstable.format.SSTableReader;
 import org.apache.cassandra.io.util.FileDataInput;
+import org.apache.cassandra.io.util.MmappedRegions;
 import org.apache.cassandra.io.util.MmappedSegmentedFile;
 import org.apache.cassandra.io.util.SegmentedFile;
 import org.apache.cassandra.schema.CachingParams;
@@ -133,7 +134,7 @@ public class SSTableReaderTest
 @Test
 public void testSpannedIndexPositions() throws IOException
 {
-MmappedSegmentedFile.MAX_SEGMENT_SIZE = 40; // each index entry is ~11 
bytes, so this will generate lots of segments
+MmappedRegions.MAX_SEGMENT_SIZE = 40; // each index entry is ~11 
bytes, so this will generate lots of segments
 
 Keyspace keyspace = Keyspace.open(KEYSPACE1);
 ColumnFamilyStore store = keyspace.getColumnFamilyStore("Standard1");

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java 
b/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
index b875a6a..4a72281 100644
--- a/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
+++ b/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
@@ -542,14 +542,12 @@ public class BufferedDataOutputStreamTest
 
 ndosp.flush();
 
-@SuppressWarnings("resource")
-ByteBufferDataInput bbdi = new 
ByteBufferDataInput(ByteBuffer.wrap(generated.toByteArray()), "", 0, 0);
-
+DataInputBuffer in = new DataInputBuffer(generated.toByteArray());
 assertEquals(expectedSize, generated.toByteArray().length);
 
 for (long v : testValues)
 {
-assertEquals(v, bbdi.readVInt());
+assertEquals(v, in.readVInt());
 }
 }
 
@@ -574,13 +572,11 @@ public class BufferedDataOutputStreamTest
 
 ndosp.flush();
 
-@SuppressWarnings("resource")
-ByteBufferDataInput bbdi = new 
ByteBufferDataInput(ByteBuffer.wrap(generated.toByteArray()), "", 0, 0);
-
+DataInputBuffer in = new DataInputBuffer(generated.toByteArray());
 assertEquals(expectedSize, generated.toByteArray().length);
 
 for (long v : testValues)
-assertEquals(v, bbdi.readUnsignedVInt());
+assertEquals(v, in.readUnsignedVInt());
 }
 
 @Test

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java 
b/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
index e051c00..364ea71 100644
--- a/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
+++ b/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
@@ -25,9 +25,7 @@ import java.io.File;
 import java.io.FileOutputStream;
 import 

[07/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
--
diff --git a/src/java/org/apache/cassandra/io/util/RandomAccessReader.java 
b/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
index b13d154..e9b0ee4 100644
--- a/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
+++ b/src/java/org/apache/cassandra/io/util/RandomAccessReader.java
@@ -19,119 +19,129 @@ package org.apache.cassandra.io.util;
 
 import java.io.*;
 import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+
+import com.google.common.primitives.Ints;
+import com.google.common.util.concurrent.RateLimiter;
 
 import org.apache.cassandra.io.FSReadError;
 import org.apache.cassandra.io.compress.BufferType;
-import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.memory.BufferPool;
 
-public class RandomAccessReader extends AbstractDataInput implements 
FileDataInput
+public class RandomAccessReader extends RebufferingInputStream implements 
FileDataInput
 {
+// The default buffer size when the client doesn't specify it
 public static final int DEFAULT_BUFFER_SIZE = 4096;
 
+// The maximum buffer size when the limiter is not null, i.e. when 
throttling
+// is enabled. This is required to avoid aquiring permits that are too 
large.
+public static final int MAX_THROTTLED_BUFFER_SIZE = 1 << 16; // 64k
+
 // the IO channel to the file, we do not own a reference to this due to
 // performance reasons (CASSANDRA-9379) so it's up to the owner of the RAR 
to
 // ensure that the channel stays open and that it is closed afterwards
 protected final ChannelProxy channel;
 
-// buffer which will cache file blocks
-protected ByteBuffer buffer;
+// optional memory mapped regions for the channel
+protected final MmappedRegions regions;
 
-// `bufferOffset` is the offset of the beginning of the buffer
-// `markedPointer` folds the offset of the last file mark
-protected long bufferOffset, markedPointer;
+// An optional limiter that will throttle the amount of data we read
+protected final RateLimiter limiter;
 
-// this can be overridden at construction to a value shorter than the true 
length of the file;
-// if so, it acts as an imposed limit on reads, rather than a convenience 
property
+// the file length, this can be overridden at construction to a value 
shorter
+// than the true length of the file; if so, it acts as an imposed limit on 
reads,
+// required when opening sstables early not to read past the mark
 private final long fileLength;
 
-protected RandomAccessReader(ChannelProxy channel, int bufferSize, long 
overrideLength, BufferType bufferType)
-{
-this.channel = channel;
+// the buffer size for buffered readers
+protected final int bufferSize;
 
-if (bufferSize <= 0)
-throw new IllegalArgumentException("bufferSize must be positive");
+// the buffer type for buffered readers
+private final BufferType bufferType;
 
-// we can cache file length in read-only mode
-fileLength = overrideLength <= 0 ? channel.size() : overrideLength;
+// offset from the beginning of the file
+protected long bufferOffset;
 
-buffer = allocateBuffer(getBufferSize(bufferSize), bufferType);
-buffer.limit(0);
-}
+// offset of the last file mark
+protected long markedPointer;
 
-/** The buffer size is typically already page aligned but if that is not 
the case
- * make sure that it is a multiple of the page size, 4096.
- * */
-protected int getBufferSize(int size)
+protected RandomAccessReader(Builder builder)
 {
-if ((size & ~4095) != size)
-{ // should already be a page size multiple but if that's not case 
round it up
-size = (size + 4095) & ~4095;
-}
-return size;
-}
+super(null);
 
-protected ByteBuffer allocateBuffer(int size, BufferType bufferType)
-{
-return BufferPool.get(size, bufferType);
-}
+this.channel = builder.channel;
+this.regions = builder.regions;
+this.limiter = builder.limiter;
+this.fileLength = builder.overrideLength <= 0 ? builder.channel.size() 
: builder.overrideLength;
+this.bufferSize = getBufferSize(builder);
+this.bufferType = builder.bufferType;
 
-// A wrapper of the RandomAccessReader that closes the channel when done.
-// For performance reasons RAR does not increase the reference count of
-// a channel but assumes the owner will keep it open and close it,
-// see CASSANDRA-9379, this thin class is just for those cases where we do
-// not have a shared channel.
-private static class RandomAccessReaderWithChannel extends 
RandomAccessReader
-{
-@SuppressWarnings("resource")
- 

[09/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

Faster sequential IO (CASSANDRA-8630)

Merge RandomAccessReader and NIODataInputStream class hierarchies
to share performance optimisation work across all readers.

patch by stefania; reviewed by ariel and benedict for CASSANDRA-8630


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce63ccc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce63ccc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce63ccc8

Branch: refs/heads/cassandra-3.0
Commit: ce63ccc842dc6e7129765391c611402eb02a3a23
Parents: 7c0bfe9
Author: Stefania Alborghetti 
Authored: Mon Jul 27 16:34:46 2015 +0800
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 12:42:35 2015 +0100

--
 build.xml   |   8 +-
 lib/licenses/ohc-0.3.4.txt  | 201 
 lib/licenses/ohc-0.4.2.txt  | 201 
 lib/ohc-core-0.4.2.jar  | Bin 0 -> 126802 bytes
 lib/ohc-core-0.4.jar| Bin 127890 -> 0 bytes
 lib/ohc-core-j8-0.4.2.jar   | Bin 0 -> 4994 bytes
 lib/ohc-core-j8-0.4.jar | Bin 4989 -> 0 bytes
 .../apache/cassandra/cache/AutoSavingCache.java |   2 +-
 .../org/apache/cassandra/cache/OHCProvider.java |   3 +-
 .../org/apache/cassandra/db/SystemKeyspace.java |   2 +-
 .../db/commitlog/CommitLogReplayer.java |  19 +-
 .../cassandra/hints/ChecksummedDataInput.java   | 128 +++--
 .../org/apache/cassandra/hints/HintsReader.java |  67 ++-
 .../compress/CompressedRandomAccessReader.java  | 147 +++---
 .../io/compress/CompressedThrottledReader.java  |  48 --
 .../io/compress/CompressionMetadata.java|   2 +-
 .../cassandra/io/sstable/KeyIterator.java   |   1 -
 .../io/sstable/format/SSTableReader.java| 100 ++--
 .../cassandra/io/sstable/format/Version.java|   2 +
 .../io/sstable/format/big/BigFormat.java|   9 +
 .../io/sstable/format/big/BigTableReader.java   | 108 ++---
 .../io/sstable/format/big/BigTableWriter.java   |   2 -
 .../cassandra/io/util/AbstractDataInput.java| 343 -
 .../io/util/BufferedSegmentedFile.java  |  24 -
 .../cassandra/io/util/ByteBufferDataInput.java  | 171 ---
 .../apache/cassandra/io/util/ChannelProxy.java  |   4 +-
 .../io/util/ChecksummedRandomAccessReader.java  |  62 ++-
 .../io/util/CompressedSegmentedFile.java| 100 ++--
 .../cassandra/io/util/DataInputBuffer.java  |  24 +-
 .../io/util/DataIntegrityMetadata.java  |  11 +-
 .../apache/cassandra/io/util/FileDataInput.java |  25 +-
 .../io/util/FileSegmentInputStream.java |  96 
 .../cassandra/io/util/ICompressedFile.java  |   9 +-
 .../cassandra/io/util/MemoryInputStream.java|  54 ++-
 .../cassandra/io/util/MmappedRegions.java   | 344 +
 .../cassandra/io/util/MmappedSegmentedFile.java | 214 +++-
 .../cassandra/io/util/NIODataInputStream.java   | 352 +-
 .../cassandra/io/util/RandomAccessReader.java   | 417 ++--
 .../io/util/RebufferingInputStream.java | 286 +++
 .../apache/cassandra/io/util/SegmentedFile.java | 115 ++---
 .../cassandra/io/util/ThrottledReader.java  |  48 --
 .../compress/CompressedStreamWriter.java|  10 +-
 .../apache/cassandra/utils/ByteBufferUtil.java  |   3 -
 .../org/apache/cassandra/utils/Throwables.java  |  45 +-
 .../apache/cassandra/utils/vint/VIntCoding.java |   2 +-
 .../cassandra/db/commitlog/CommitLogTest.java   |   8 +-
 .../db/commitlog/CommitLogTestReplayer.java |   3 +-
 .../hints/ChecksummedDataInputTest.java | 227 +++--
 .../io/ChecksummedRandomAccessReaderTest.java   | 127 -
 .../cassandra/io/RandomAccessReaderTest.java| 269 ---
 .../CompressedRandomAccessReaderTest.java   | 245 +-
 .../CompressedSequentialWriterTest.java |   2 +-
 .../cassandra/io/sstable/SSTableReaderTest.java |   3 +-
 .../io/util/BufferedDataOutputStreamTest.java   |  12 +-
 .../io/util/BufferedRandomAccessFileTest.java   |  91 +---
 .../util/ChecksummedRandomAccessReaderTest.java | 127 +
 .../io/util/FileSegmentInputStreamTest.java | 131 +
 .../apache/cassandra/io/util/MemoryTest.java|  37 ++
 .../cassandra/io/util/MmappedRegionsTest.java   | 375 ++
 .../io/util/NIODataInputStreamTest.java |  16 +-
 .../io/util/RandomAccessReaderTest.java | 483 +++
 61 files changed, 3339 insertions(+), 2626 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/build.xml
--
diff --git a/build.xml b/build.xml
index 28f08d6..252b1a8 100644
--- a/build.xml
+++ 

[04/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/src/java/org/apache/cassandra/io/util/AbstractDataInput.java
--
diff --git a/src/java/org/apache/cassandra/io/util/AbstractDataInput.java 
b/src/java/org/apache/cassandra/io/util/AbstractDataInput.java
deleted file mode 100644
index 29ce2c3..000
--- a/src/java/org/apache/cassandra/io/util/AbstractDataInput.java
+++ /dev/null
@@ -1,343 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.io.util;
-
-import java.io.*;
-
-public abstract class AbstractDataInput extends InputStream implements 
DataInputPlus
-{
-public abstract void seek(long position) throws IOException;
-public abstract long getPosition();
-public abstract long getPositionLimit();
-
-public int skipBytes(int n) throws IOException
-{
-if (n <= 0)
-return 0;
-long oldPosition = getPosition();
-seek(Math.min(getPositionLimit(), oldPosition + n));
-long skipped = getPosition() - oldPosition;
-assert skipped >= 0 && skipped <= n;
-return (int) skipped;
-}
-
-/**
- * Reads a boolean from the current position in this file. Blocks until one
- * byte has been read, the end of the file is reached or an exception is
- * thrown.
- *
- * @return the next boolean value from this file.
- * @throws java.io.EOFException
- * if the end of this file is detected.
- * @throws java.io.IOException
- * if this file is closed or another I/O error occurs.
- */
-public final boolean readBoolean() throws IOException {
-int temp = this.read();
-if (temp < 0) {
-throw new EOFException();
-}
-return temp != 0;
-}
-
-/**
- * Reads an 8-bit byte from the current position in this file. Blocks until
- * one byte has been read, the end of the file is reached or an exception 
is
- * thrown.
- *
- * @return the next signed 8-bit byte value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-public final byte readByte() throws IOException {
-int temp = this.read();
-if (temp < 0) {
-throw new EOFException();
-}
-return (byte) temp;
-}
-
-/**
- * Reads a 16-bit character from the current position in this file. Blocks 
until
- * two bytes have been read, the end of the file is reached or an 
exception is
- * thrown.
- *
- * @return the next char value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-public final char readChar() throws IOException {
-int ch1 = this.read();
-int ch2 = this.read();
-if ((ch1 | ch2) < 0)
-throw new EOFException();
-return (char)((ch1 << 8) + (ch2 << 0));
-}
-
-/**
- * Reads a 64-bit double from the current position in this file. Blocks
- * until eight bytes have been read, the end of the file is reached or an
- * exception is thrown.
- *
- * @return the next double value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-public final double readDouble() throws IOException {
-return Double.longBitsToDouble(readLong());
-}
-
-/**
- * Reads a 32-bit float from the current position in this file. Blocks
- * until four bytes have been read, the end of the file is reached or an
- * exception is thrown.
- *
- * @return the next float value from this file.
- * @throws EOFException
- * if the end of this file is detected.
- * @throws IOException
- * if this file is closed or another I/O error occurs.
- */
-  

[05/10] cassandra git commit: Faster sequential IO (CASSANDRA-8630)

[benedict]

Faster sequential IO (CASSANDRA-8630)

Merge RandomAccessReader and NIODataInputStream class hierarchies
to share performance optimisation work across all readers.

patch by stefania; reviewed by ariel and benedict for CASSANDRA-8630


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce63ccc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce63ccc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce63ccc8

Branch: refs/heads/trunk
Commit: ce63ccc842dc6e7129765391c611402eb02a3a23
Parents: 7c0bfe9
Author: Stefania Alborghetti 
Authored: Mon Jul 27 16:34:46 2015 +0800
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 12:42:35 2015 +0100

--
 build.xml   |   8 +-
 lib/licenses/ohc-0.3.4.txt  | 201 
 lib/licenses/ohc-0.4.2.txt  | 201 
 lib/ohc-core-0.4.2.jar  | Bin 0 -> 126802 bytes
 lib/ohc-core-0.4.jar| Bin 127890 -> 0 bytes
 lib/ohc-core-j8-0.4.2.jar   | Bin 0 -> 4994 bytes
 lib/ohc-core-j8-0.4.jar | Bin 4989 -> 0 bytes
 .../apache/cassandra/cache/AutoSavingCache.java |   2 +-
 .../org/apache/cassandra/cache/OHCProvider.java |   3 +-
 .../org/apache/cassandra/db/SystemKeyspace.java |   2 +-
 .../db/commitlog/CommitLogReplayer.java |  19 +-
 .../cassandra/hints/ChecksummedDataInput.java   | 128 +++--
 .../org/apache/cassandra/hints/HintsReader.java |  67 ++-
 .../compress/CompressedRandomAccessReader.java  | 147 +++---
 .../io/compress/CompressedThrottledReader.java  |  48 --
 .../io/compress/CompressionMetadata.java|   2 +-
 .../cassandra/io/sstable/KeyIterator.java   |   1 -
 .../io/sstable/format/SSTableReader.java| 100 ++--
 .../cassandra/io/sstable/format/Version.java|   2 +
 .../io/sstable/format/big/BigFormat.java|   9 +
 .../io/sstable/format/big/BigTableReader.java   | 108 ++---
 .../io/sstable/format/big/BigTableWriter.java   |   2 -
 .../cassandra/io/util/AbstractDataInput.java| 343 -
 .../io/util/BufferedSegmentedFile.java  |  24 -
 .../cassandra/io/util/ByteBufferDataInput.java  | 171 ---
 .../apache/cassandra/io/util/ChannelProxy.java  |   4 +-
 .../io/util/ChecksummedRandomAccessReader.java  |  62 ++-
 .../io/util/CompressedSegmentedFile.java| 100 ++--
 .../cassandra/io/util/DataInputBuffer.java  |  24 +-
 .../io/util/DataIntegrityMetadata.java  |  11 +-
 .../apache/cassandra/io/util/FileDataInput.java |  25 +-
 .../io/util/FileSegmentInputStream.java |  96 
 .../cassandra/io/util/ICompressedFile.java  |   9 +-
 .../cassandra/io/util/MemoryInputStream.java|  54 ++-
 .../cassandra/io/util/MmappedRegions.java   | 344 +
 .../cassandra/io/util/MmappedSegmentedFile.java | 214 +++-
 .../cassandra/io/util/NIODataInputStream.java   | 352 +-
 .../cassandra/io/util/RandomAccessReader.java   | 417 ++--
 .../io/util/RebufferingInputStream.java | 286 +++
 .../apache/cassandra/io/util/SegmentedFile.java | 115 ++---
 .../cassandra/io/util/ThrottledReader.java  |  48 --
 .../compress/CompressedStreamWriter.java|  10 +-
 .../apache/cassandra/utils/ByteBufferUtil.java  |   3 -
 .../org/apache/cassandra/utils/Throwables.java  |  45 +-
 .../apache/cassandra/utils/vint/VIntCoding.java |   2 +-
 .../cassandra/db/commitlog/CommitLogTest.java   |   8 +-
 .../db/commitlog/CommitLogTestReplayer.java |   3 +-
 .../hints/ChecksummedDataInputTest.java | 227 +++--
 .../io/ChecksummedRandomAccessReaderTest.java   | 127 -
 .../cassandra/io/RandomAccessReaderTest.java| 269 ---
 .../CompressedRandomAccessReaderTest.java   | 245 +-
 .../CompressedSequentialWriterTest.java |   2 +-
 .../cassandra/io/sstable/SSTableReaderTest.java |   3 +-
 .../io/util/BufferedDataOutputStreamTest.java   |  12 +-
 .../io/util/BufferedRandomAccessFileTest.java   |  91 +---
 .../util/ChecksummedRandomAccessReaderTest.java | 127 +
 .../io/util/FileSegmentInputStreamTest.java | 131 +
 .../apache/cassandra/io/util/MemoryTest.java|  37 ++
 .../cassandra/io/util/MmappedRegionsTest.java   | 375 ++
 .../io/util/NIODataInputStreamTest.java |  16 +-
 .../io/util/RandomAccessReaderTest.java | 483 +++
 61 files changed, 3339 insertions(+), 2626 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce63ccc8/build.xml
--
diff --git a/build.xml b/build.xml
index 28f08d6..252b1a8 100644
--- a/build.xml
+++ 

[1/3] cassandra git commit: Fix startup error when upgrading nodes to 3.0

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 083815e4f -> eaf0e3523


Fix startup error when upgrading nodes to 3.0

patch by slebresne; reviewed by blambov for CASSANDRA-10136


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f009272b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f009272b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f009272b

Branch: refs/heads/trunk
Commit: f009272ba6ffb94579a443733f5cfe25ec9606fa
Parents: dc5b268
Author: Sylvain Lebresne 
Authored: Wed Aug 19 16:13:52 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:11:42 2015 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/Serializers.java| 65 ++--
 .../db/SinglePartitionNamesCommand.java |  2 +-
 .../columniterator/AbstractSSTableIterator.java | 36 +--
 .../db/columniterator/SSTableIterator.java  |  4 +-
 .../columniterator/SSTableReversedIterator.java | 35 +--
 .../cassandra/db/marshal/CompositeType.java |  5 ++
 .../cassandra/io/sstable/IndexHelper.java   |  6 +-
 8 files changed, 133 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f009272b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fb33de0..726eb04 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-beta2
+ * Fix startup error when upgrading nodes (CASSANDRA-10136)
  * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
  * Improve batchlog write patch (CASSANDRA-9673)
  * Re-apply MaterializedView updates on commitlog replay (CASSANDRA-10164)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f009272b/src/java/org/apache/cassandra/db/Serializers.java
--
diff --git a/src/java/org/apache/cassandra/db/Serializers.java 
b/src/java/org/apache/cassandra/db/Serializers.java
index 8056222..2561bbe 100644
--- a/src/java/org/apache/cassandra/db/Serializers.java
+++ b/src/java/org/apache/cassandra/db/Serializers.java
@@ -18,12 +18,16 @@
 package org.apache.cassandra.db;
 
 import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.List;
 
 import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.db.marshal.CompositeType;
 import org.apache.cassandra.io.ISerializer;
 import org.apache.cassandra.io.util.DataInputPlus;
 import org.apache.cassandra.io.util.DataOutputPlus;
 import org.apache.cassandra.io.sstable.format.Version;
+import org.apache.cassandra.utils.ByteBufferUtil;
 
 import static org.apache.cassandra.io.sstable.IndexHelper.IndexInfo;
 
@@ -44,12 +48,65 @@ public class Serializers
 return new IndexInfo.Serializer(metadata, version);
 }
 
-// Note that for the old layout, this will actually discard the cellname 
parts that are not strictly
-// part of the clustering prefix. Don't use this if that's not what you 
want.
-public ISerializer clusteringPrefixSerializer(final 
Version version, final SerializationHeader header)
+// TODO: Once we drop support for old (pre-3.0) sstables, we can drop this 
method and inline the calls to
+// ClusteringPrefix.serializer in IndexHelper directly. At which point 
this whole class probably becomes
+// unecessary (since IndexInfo.Serializer won't depend on the metadata 
either).
+public ISerializer 
indexEntryClusteringPrefixSerializer(final Version version, final 
SerializationHeader header)
 {
 if (!version.storeRows())
-throw new UnsupportedOperationException();
+{
+return new ISerializer()
+{
+public void serialize(ClusteringPrefix clustering, 
DataOutputPlus out) throws IOException
+{
+// We should only use this for reading old sstable, never 
write new ones.
+throw new UnsupportedOperationException();
+}
+
+public ClusteringPrefix deserialize(DataInputPlus in) throws 
IOException
+{
+// We're reading the old cellname/composite
+ByteBuffer bb = ByteBufferUtil.readWithShortLength(in);
+assert bb.hasRemaining(); // empty cellnames were invalid
+
+int clusteringSize = metadata.clusteringColumns().size();
+// If the table has no clustering column, then the 
cellname will just be the "column" name, which we ignore here.
+if (clusteringSize == 0)
+return Clustering.EMPTY;
+
+if 

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[slebresne]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eaf0e352
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eaf0e352
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eaf0e352

Branch: refs/heads/trunk
Commit: eaf0e35233babae6176a73439a38bb0e866c64e8
Parents: 083815e 24682d2
Author: Sylvain Lebresne 
Authored: Fri Sep 4 11:16:23 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 11:16:23 2015 +0200

--
 CHANGES.txt |  2 +
 .../org/apache/cassandra/db/ReadResponse.java   | 38 +++-
 .../org/apache/cassandra/db/Serializers.java| 65 -
 .../db/SinglePartitionNamesCommand.java |  2 +-
 .../columniterator/AbstractSSTableIterator.java | 36 ++-
 .../db/columniterator/SSTableIterator.java  |  4 +-
 .../columniterator/SSTableReversedIterator.java | 35 +--
 .../cassandra/db/marshal/CompositeType.java |  5 +
 .../cassandra/io/sstable/IndexHelper.java   |  6 +-
 .../apache/cassandra/db/ReadResponseTest.java   | 99 
 10 files changed, 266 insertions(+), 26 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/eaf0e352/CHANGES.txt
--
diff --cc CHANGES.txt
index ef6509c,4d8a932..84eed2a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,6 +1,10 @@@
 +3.2
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0.0-beta2
+  * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
+  * Fix startup error when upgrading nodes (CASSANDRA-10136)
   * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
   * Improve batchlog write patch (CASSANDRA-9673)
   * Re-apply MaterializedView updates on commitlog replay (CASSANDRA-10164)

[jira] [Commented] (CASSANDRA-9975) Flatten RowIterator call hierarchy with a shared RowTransformer

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730571#comment-14730571
 ] 

Benedict commented on CASSANDRA-9975:
-

I've pushed an update that fixes a problem with {{IllegalStateException}} being 
thrown erroneously. I had previously provided {{UnfilteredRowFunction}} as the 
default function type as I thought this would be easier, and provided 
{{RowFunction}} as a convenience for clients that operated only an 
{{RowIterator}} - however I forgot this myself, and used it for an 
{{UnfilteredRowIterator}}, which it was rejecting as invalid.

Instead I've split {{UnfilteredRowFunction}} into {{RowFunction}} and 
{{MarkerFunction}}, and rejected the latter in the constructor to a 
{{TransfprmingRowIterator}}.

I've also reintroduced {{BTreeRow.isLive}} - not sure how that got removed - 
and swapped the constructor argument order for {{TransformingIterator}} to 
avoid class cast conflicts.

> Flatten RowIterator call hierarchy with a shared RowTransformer
> ---
>
> Key: CASSANDRA-9975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9975
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> Stepping through a read response is made exceedingly difficult by the sheer 
> depth of the call hierarchy, and how rapidly your context jumps around. This 
> ticket intend to partially address that, by flattening one of the main causes 
> of this: iterator transformations.
> I have a patch that attempts to mitigate (but not entirely eliminate) this, 
> through the introduction of a {{RowTransformer}} class that all 
> transformations are applied through. If a transformation has already been 
> applied, the {{RowTransformer}} class does not wrap a new iterator, but 
> instead returns a new {{RowTransformer}} that wraps the original underlying 
> (untransformed) iterator and both transformations. This can accumulate an 
> arbitrary number of transformations and, quite importantly, can apply the 
> filtration step {{Unfiltered -> Row}}  in the same instance as well. The 
> intention being that a majority of control flow happens inside this 
> {{RowTransformer}}, so there is far less context jumping to cope with.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10220) Memtables do not handle column changes for their PartitionColumns

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730632#comment-14730632
 ] 

Sylvain Lebresne commented on CASSANDRA-10220:
--

For info, I've pushed a rebased version with 2 additional commits 
[here|https://github.com/pcmanus/cassandra/commits/10220]. It does the 
following changes to the original branch:
* when calling {{AbstractBTreePartition.unfilteredIterator(ColumnFilter 
selection, ...)}}, the resulting iterator {{columns()}} was not returning the 
original partition columns, even though it was guaranteed to only contain 
columns from {{selection}} and hence calls would expect the iterator 
{{columns()}} method to return either {{selection.fetchedColumns()}} or a 
subset of it, which wasn't the case. I'll note that I don't know if this was 
really manifesting as a bug, but I still think it's worth changing. Also, as I 
mention in the commit, the ideal would be to take the intersection of the the 
filter and the partition columns, but while that would be more precise, I don't 
think we rely enough on such precision that it's worth making that computation.
* In {{AtomicBtreePartitions.addAllWithSizeDelta()}}, the columns from the 
update were not merged with the existing ones, which I suspect was an oversight.
* For {{PartitionUpdate}}, the code actually passes the columns in the ctor 
(and we even validate any added row does only has columns from this set), so I 
think the right thing to do is just to return that and that's what the last 
commit does.

I'll note that I appear to have been wrong about CASSANDRA-10169, this doesn't 
seem to fix it. So I'll wait on CI to run on this branch before calling this 
"ready for review" and I'll continue looking at CASSANDRA-10169 in the meantime.


> Memtables do not handle column changes for their PartitionColumns
> -
>
> Key: CASSANDRA-10220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> There are a couple of inherited issues around the {{PartitionColumns}} in 
> {{AbstractBTreePartition}}. Firstly, we do not always return these columns, 
> but sometimes the metadata columns. Secondly, the columns cannot be updated 
> with new columns, and the metadata columns will not show dropped columns. 
> This should most likely be moved into the {{Holder}}, and updated along with 
> any other updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10193) Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to Collection

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730722#comment-14730722
 ] 

Marcus Eriksson commented on CASSANDRA-10193:
-

OK, +1

> Improve Rows.diff and Rows.merge efficiency; downgrade Row.columns() to 
> Collection
> 
>
> Key: CASSANDRA-10193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10193
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.0 rc1
>
>
> There's not really a lot of reason to store a Columns instance in each row. 
> Retaining it introduces extra costs on every row merge, in both consumed CPU 
> time and heap. 
> While working on CASSANDRA-10045 it became apparent this would be very easy 
> to remove, however to avoid scope creep I have filed this as a follow up 
> ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Improve efficiency of Row

[benedict]

Improve efficiency of Row

Removes need to store a Columns instance within a Row,
avoiding both the cost of merging and storing these.

Improves algorithmic efficiency of Rows.merge by performing
a straight forward linear merge on the present ColumnData.

Improves algorithmic efficiency of Rows.diff by utilising MergeIterator

patch by benedict; reviewed by marcus for CASSANDRA-10193


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aa576263
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aa576263
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aa576263

Branch: refs/heads/trunk
Commit: aa5762633a06e229cd9122983eec94bdb74fc23a
Parents: ce63ccc
Author: Benedict Elliott Smith 
Authored: Wed Aug 26 19:02:03 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 13:47:07 2015 +0100

--
 .../apache/cassandra/cql3/UpdateParameters.java |   4 +-
 src/java/org/apache/cassandra/db/Columns.java   |   2 +-
 .../org/apache/cassandra/db/LegacyLayout.java   |   4 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   |   6 +-
 .../cassandra/db/SerializationHeader.java   |   1 -
 .../cassandra/db/UnfilteredDeserializer.java|   2 +-
 .../db/compaction/CompactionIterator.java   |   4 +-
 .../db/partitions/AtomicBTreePartition.java |   8 +-
 .../db/partitions/PartitionUpdate.java  |  10 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   1 -
 .../org/apache/cassandra/db/rows/BTreeRow.java  |  64 +++---
 src/java/org/apache/cassandra/db/rows/Row.java  |  15 +-
 src/java/org/apache/cassandra/db/rows/Rows.java | 214 ++-
 .../rows/UnfilteredRowIteratorSerializer.java   |   2 +-
 .../db/rows/UnfilteredRowIterators.java |  21 +-
 .../cassandra/db/rows/UnfilteredSerializer.java |   2 +-
 .../cassandra/db/view/MaterializedView.java |   6 +-
 .../cassandra/index/SecondaryIndexManager.java  |  12 +-
 .../transactions/CompactionTransaction.java |   5 +-
 .../io/sstable/SSTableSimpleIterator.java   |   2 +-
 .../apache/cassandra/service/DataResolver.java  |   6 +-
 .../cassandra/thrift/ThriftResultsMerger.java   |   6 +-
 .../utils/memory/AbstractAllocator.java |   8 +-
 .../utils/memory/MemtableAllocator.java |   2 +-
 .../utils/memory/MemtableBufferAllocator.java   |   5 +-
 .../cassandra/utils/memory/NativeAllocator.java |   2 +-
 test/unit/org/apache/cassandra/db/RowTest.java  |   2 +-
 .../cassandra/triggers/TriggerExecutorTest.java |   2 +-
 28 files changed, 210 insertions(+), 208 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa576263/src/java/org/apache/cassandra/cql3/UpdateParameters.java
--
diff --git a/src/java/org/apache/cassandra/cql3/UpdateParameters.java 
b/src/java/org/apache/cassandra/cql3/UpdateParameters.java
index 22aeddd..cd81f84 100644
--- a/src/java/org/apache/cassandra/cql3/UpdateParameters.java
+++ b/src/java/org/apache/cassandra/cql3/UpdateParameters.java
@@ -121,13 +121,13 @@ public class UpdateParameters
 if (clustering == Clustering.STATIC_CLUSTERING)
 {
 if (staticBuilder == null)
-staticBuilder = 
BTreeRow.unsortedBuilder(updatedColumns.statics, nowInSec);
+staticBuilder = BTreeRow.unsortedBuilder(nowInSec);
 builder = staticBuilder;
 }
 else
 {
 if (regularBuilder == null)
-regularBuilder = 
BTreeRow.unsortedBuilder(updatedColumns.regulars, nowInSec);
+regularBuilder = BTreeRow.unsortedBuilder(nowInSec);
 builder = regularBuilder;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa576263/src/java/org/apache/cassandra/db/Columns.java
--
diff --git a/src/java/org/apache/cassandra/db/Columns.java 
b/src/java/org/apache/cassandra/db/Columns.java
index 46e8401..582b742 100644
--- a/src/java/org/apache/cassandra/db/Columns.java
+++ b/src/java/org/apache/cassandra/db/Columns.java
@@ -86,7 +86,7 @@ public class Columns extends 
AbstractCollection implements Col
  * @param s the set from which to create the new {@code Columns}.
  * @return the newly created {@code Columns} containing the columns from 
{@code s}.
  */
-public static Columns from(Set s)
+public static Columns from(Collection s)
 {
 Object[] tree = 
BTree.builder(Comparator.naturalOrder()).addAll(s).build();
 return new Columns(tree, findFirstComplexIdx(tree));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa576263/src/java/org/apache/cassandra/db/LegacyLayout.java

[1/3] cassandra git commit: ninja fix comments and remove unused method

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 aa5762633 -> a22db5be2
  refs/heads/trunk ae89731f0 -> ae9914bb5


ninja fix comments and remove unused method


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a22db5be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a22db5be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a22db5be

Branch: refs/heads/cassandra-3.0
Commit: a22db5be2d07fddd6529d8a11a5409593af6e4c2
Parents: aa57626
Author: Benedict Elliott Smith 
Authored: Fri Sep 4 13:56:07 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 13:56:07 2015 +0100

--
 src/java/org/apache/cassandra/db/rows/BTreeRow.java | 5 -
 src/java/org/apache/cassandra/db/rows/Row.java  | 9 ++---
 2 files changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a22db5be/src/java/org/apache/cassandra/db/rows/BTreeRow.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/BTreeRow.java 
b/src/java/org/apache/cassandra/db/rows/BTreeRow.java
index 653ffcd..af09cb5 100644
--- a/src/java/org/apache/cassandra/db/rows/BTreeRow.java
+++ b/src/java/org/apache/cassandra/db/rows/BTreeRow.java
@@ -164,11 +164,6 @@ public class BTreeRow extends AbstractRow
 return Collections2.transform(this, ColumnData::column);
 }
 
-public Collection actualColumns()
-{
-return Collections2.transform(this, ColumnData::column);
-}
-
 public LivenessInfo primaryKeyLivenessInfo()
 {
 return primaryKeyLivenessInfo;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a22db5be/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/Row.java 
b/src/java/org/apache/cassandra/db/rows/Row.java
index 1b18b38..a351306 100644
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@ -52,13 +52,8 @@ public interface Row extends Unfiltered, 
Collection
 public Clustering clustering();
 
 /**
- * The columns this row contains.
- *
- * Note that this is actually a superset of the columns the row contains. 
The row
- * may not have values for each of those columns, but it can't have values 
for other
- * columns.
- *
- * @return a superset of the columns contained in this row.
+ * An in-natural-order collection of the columns for which data (incl. 
simple tombstones)
+ * is present in this row.
  */
 public Collection columns();
 

[jira] [Commented] (CASSANDRA-10262) Bootstrapping nodes failing to apply view updates correctly.

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730746#comment-14730746
 ] 

T Jake Luciani commented on CASSANDRA-10262:


That change didn't work, looks like the code sets joined = true even if it's 
bootstrapping.  Checking getOperationMode() or using my version would be better.


> Bootstrapping nodes failing to apply view updates correctly.
> 
>
> Key: CASSANDRA-10262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10262
> Project: Cassandra
>  Issue Type: Bug
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.0.0 rc1
>
>
> The bootstrapping dtests started failing after CASSANDRA-10164
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/130/testReport/junit/materialized_views_test/TestMaterializedViews/add_node_after_mv_test/
> The underlying issue is we were never applying MV updates to non-joined nodes 
> before. Now that we are we can see bootstrapping nodes need to handle pending 
> nodes when calculating the paired replica in the case there is no natural 
> endpoints for that node (since it is pending).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: ninja fix comments and remove unused method

[benedict]

ninja fix comments and remove unused method


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a22db5be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a22db5be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a22db5be

Branch: refs/heads/trunk
Commit: a22db5be2d07fddd6529d8a11a5409593af6e4c2
Parents: aa57626
Author: Benedict Elliott Smith 
Authored: Fri Sep 4 13:56:07 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 13:56:07 2015 +0100

--
 src/java/org/apache/cassandra/db/rows/BTreeRow.java | 5 -
 src/java/org/apache/cassandra/db/rows/Row.java  | 9 ++---
 2 files changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a22db5be/src/java/org/apache/cassandra/db/rows/BTreeRow.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/BTreeRow.java 
b/src/java/org/apache/cassandra/db/rows/BTreeRow.java
index 653ffcd..af09cb5 100644
--- a/src/java/org/apache/cassandra/db/rows/BTreeRow.java
+++ b/src/java/org/apache/cassandra/db/rows/BTreeRow.java
@@ -164,11 +164,6 @@ public class BTreeRow extends AbstractRow
 return Collections2.transform(this, ColumnData::column);
 }
 
-public Collection actualColumns()
-{
-return Collections2.transform(this, ColumnData::column);
-}
-
 public LivenessInfo primaryKeyLivenessInfo()
 {
 return primaryKeyLivenessInfo;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a22db5be/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/Row.java 
b/src/java/org/apache/cassandra/db/rows/Row.java
index 1b18b38..a351306 100644
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@ -52,13 +52,8 @@ public interface Row extends Unfiltered, 
Collection
 public Clustering clustering();
 
 /**
- * The columns this row contains.
- *
- * Note that this is actually a superset of the columns the row contains. 
The row
- * may not have values for each of those columns, but it can't have values 
for other
- * columns.
- *
- * @return a superset of the columns contained in this row.
+ * An in-natural-order collection of the columns for which data (incl. 
simple tombstones)
+ * is present in this row.
  */
 public Collection columns();
 

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[benedict]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae9914bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae9914bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae9914bb

Branch: refs/heads/trunk
Commit: ae9914bb5e8d51c1a331d78249d5706bd35d70cc
Parents: ae89731 a22db5b
Author: Benedict Elliott Smith 
Authored: Fri Sep 4 13:56:16 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 13:56:16 2015 +0100

--
 src/java/org/apache/cassandra/db/rows/BTreeRow.java | 5 -
 src/java/org/apache/cassandra/db/rows/Row.java  | 9 ++---
 2 files changed, 2 insertions(+), 12 deletions(-)
--

[jira] [Commented] (CASSANDRA-10169) Test regression for consistency_test.TestAccuracy

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730758#comment-14730758
 ] 

Sylvain Lebresne commented on CASSANDRA-10169:
--

Ok, so the problem was that the row returned by {{BtreeRow.filter(ColumnFilter 
selection)}} had the same {{Columns}} than pre-filtering, and that was a case 
where the ended up making the whole iterator advertising more columns that it 
should have, which was breaking things later on. But as it happens, this is 
fixed by the just committed patch for CASSANDRA-10193 (in the case in question, 
the actual columns of the rows are used which is the right thing to do). I've 
manually checked that the test in question passes now so closing as duplicate 
of CASSANDRA-10193.

> Test regression for consistency_test.TestAccuracy
> -
>
> Key: CASSANDRA-10169
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10169
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> We have 2 tests failures:
> {noformat}
> consistency_test.TestAccuracy.test_network_topology_strategy_users
> consistency_test.TestAccuracy.test_simple_strategy_users
> {noformat}
> and looking at [the 
> history|http://cassci.datastax.com/job/cassandra-3.0_dtest/90/testReport/junit/consistency_test/TestAccuracy/test_simple_strategy_users/history/?start=50],
>  CASSANDRA-9913 appears to be the culprit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10169) Test regression for consistency_test.TestAccuracy

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-10169.
--
Resolution: Duplicate

> Test regression for consistency_test.TestAccuracy
> -
>
> Key: CASSANDRA-10169
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10169
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> We have 2 tests failures:
> {noformat}
> consistency_test.TestAccuracy.test_network_topology_strategy_users
> consistency_test.TestAccuracy.test_simple_strategy_users
> {noformat}
> and looking at [the 
> history|http://cassci.datastax.com/job/cassandra-3.0_dtest/90/testReport/junit/consistency_test/TestAccuracy/test_simple_strategy_users/history/?start=50],
>  CASSANDRA-9913 appears to be the culprit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[benedict]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae89731f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae89731f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae89731f

Branch: refs/heads/trunk
Commit: ae89731f07020f3ce921fbf96a5adb424ac4f9c5
Parents: 356dd5b aa57626
Author: Benedict Elliott Smith 
Authored: Fri Sep 4 13:47:22 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 13:47:22 2015 +0100

--
 .../apache/cassandra/cql3/UpdateParameters.java |   4 +-
 src/java/org/apache/cassandra/db/Columns.java   |   2 +-
 .../org/apache/cassandra/db/LegacyLayout.java   |   4 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   |   6 +-
 .../cassandra/db/SerializationHeader.java   |   1 -
 .../cassandra/db/UnfilteredDeserializer.java|   2 +-
 .../db/compaction/CompactionIterator.java   |   4 +-
 .../db/partitions/AtomicBTreePartition.java |   8 +-
 .../db/partitions/PartitionUpdate.java  |  10 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   1 -
 .../org/apache/cassandra/db/rows/BTreeRow.java  |  64 +++---
 src/java/org/apache/cassandra/db/rows/Row.java  |  15 +-
 src/java/org/apache/cassandra/db/rows/Rows.java | 214 ++-
 .../rows/UnfilteredRowIteratorSerializer.java   |   2 +-
 .../db/rows/UnfilteredRowIterators.java |  21 +-
 .../cassandra/db/rows/UnfilteredSerializer.java |   2 +-
 .../cassandra/db/view/MaterializedView.java |   6 +-
 .../cassandra/index/SecondaryIndexManager.java  |  12 +-
 .../transactions/CompactionTransaction.java |   5 +-
 .../io/sstable/SSTableSimpleIterator.java   |   2 +-
 .../apache/cassandra/service/DataResolver.java  |   6 +-
 .../cassandra/thrift/ThriftResultsMerger.java   |   6 +-
 .../utils/memory/AbstractAllocator.java |   8 +-
 .../utils/memory/MemtableAllocator.java |   2 +-
 .../utils/memory/MemtableBufferAllocator.java   |   5 +-
 .../cassandra/utils/memory/NativeAllocator.java |   2 +-
 test/unit/org/apache/cassandra/db/RowTest.java  |   2 +-
 .../cassandra/triggers/TriggerExecutorTest.java |   2 +-
 28 files changed, 210 insertions(+), 208 deletions(-)
--

[1/3] cassandra git commit: Improve efficiency of Row

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 ce63ccc84 -> aa5762633
  refs/heads/trunk 356dd5b31 -> ae89731f0


Improve efficiency of Row

Removes need to store a Columns instance within a Row,
avoiding both the cost of merging and storing these.

Improves algorithmic efficiency of Rows.merge by performing
a straight forward linear merge on the present ColumnData.

Improves algorithmic efficiency of Rows.diff by utilising MergeIterator

patch by benedict; reviewed by marcus for CASSANDRA-10193


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aa576263
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aa576263
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aa576263

Branch: refs/heads/cassandra-3.0
Commit: aa5762633a06e229cd9122983eec94bdb74fc23a
Parents: ce63ccc
Author: Benedict Elliott Smith 
Authored: Wed Aug 26 19:02:03 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Fri Sep 4 13:47:07 2015 +0100

--
 .../apache/cassandra/cql3/UpdateParameters.java |   4 +-
 src/java/org/apache/cassandra/db/Columns.java   |   2 +-
 .../org/apache/cassandra/db/LegacyLayout.java   |   4 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   |   6 +-
 .../cassandra/db/SerializationHeader.java   |   1 -
 .../cassandra/db/UnfilteredDeserializer.java|   2 +-
 .../db/compaction/CompactionIterator.java   |   4 +-
 .../db/partitions/AtomicBTreePartition.java |   8 +-
 .../db/partitions/PartitionUpdate.java  |  10 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   1 -
 .../org/apache/cassandra/db/rows/BTreeRow.java  |  64 +++---
 src/java/org/apache/cassandra/db/rows/Row.java  |  15 +-
 src/java/org/apache/cassandra/db/rows/Rows.java | 214 ++-
 .../rows/UnfilteredRowIteratorSerializer.java   |   2 +-
 .../db/rows/UnfilteredRowIterators.java |  21 +-
 .../cassandra/db/rows/UnfilteredSerializer.java |   2 +-
 .../cassandra/db/view/MaterializedView.java |   6 +-
 .../cassandra/index/SecondaryIndexManager.java  |  12 +-
 .../transactions/CompactionTransaction.java |   5 +-
 .../io/sstable/SSTableSimpleIterator.java   |   2 +-
 .../apache/cassandra/service/DataResolver.java  |   6 +-
 .../cassandra/thrift/ThriftResultsMerger.java   |   6 +-
 .../utils/memory/AbstractAllocator.java |   8 +-
 .../utils/memory/MemtableAllocator.java |   2 +-
 .../utils/memory/MemtableBufferAllocator.java   |   5 +-
 .../cassandra/utils/memory/NativeAllocator.java |   2 +-
 test/unit/org/apache/cassandra/db/RowTest.java  |   2 +-
 .../cassandra/triggers/TriggerExecutorTest.java |   2 +-
 28 files changed, 210 insertions(+), 208 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa576263/src/java/org/apache/cassandra/cql3/UpdateParameters.java
--
diff --git a/src/java/org/apache/cassandra/cql3/UpdateParameters.java 
b/src/java/org/apache/cassandra/cql3/UpdateParameters.java
index 22aeddd..cd81f84 100644
--- a/src/java/org/apache/cassandra/cql3/UpdateParameters.java
+++ b/src/java/org/apache/cassandra/cql3/UpdateParameters.java
@@ -121,13 +121,13 @@ public class UpdateParameters
 if (clustering == Clustering.STATIC_CLUSTERING)
 {
 if (staticBuilder == null)
-staticBuilder = 
BTreeRow.unsortedBuilder(updatedColumns.statics, nowInSec);
+staticBuilder = BTreeRow.unsortedBuilder(nowInSec);
 builder = staticBuilder;
 }
 else
 {
 if (regularBuilder == null)
-regularBuilder = 
BTreeRow.unsortedBuilder(updatedColumns.regulars, nowInSec);
+regularBuilder = BTreeRow.unsortedBuilder(nowInSec);
 builder = regularBuilder;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa576263/src/java/org/apache/cassandra/db/Columns.java
--
diff --git a/src/java/org/apache/cassandra/db/Columns.java 
b/src/java/org/apache/cassandra/db/Columns.java
index 46e8401..582b742 100644
--- a/src/java/org/apache/cassandra/db/Columns.java
+++ b/src/java/org/apache/cassandra/db/Columns.java
@@ -86,7 +86,7 @@ public class Columns extends 
AbstractCollection implements Col
  * @param s the set from which to create the new {@code Columns}.
  * @return the newly created {@code Columns} containing the columns from 
{@code s}.
  */
-public static Columns from(Set s)
+public static Columns from(Collection s)
 {
 Object[] tree = 
BTree.builder(Comparator.naturalOrder()).addAll(s).build();
 return new Columns(tree, findFirstComplexIdx(tree));

cassandra git commit: fix

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 a22db5be2 -> c7557bdec


fix


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7557bde
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7557bde
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7557bde

Branch: refs/heads/cassandra-3.0
Commit: c7557bdecda6ffe76bb5ee62be4c2d6c53804681
Parents: a22db5b
Author: T Jake Luciani 
Authored: Thu Sep 3 14:53:00 2015 -0400
Committer: T Jake Luciani 
Committed: Fri Sep 4 09:10:49 2015 -0400

--
 .../cassandra/db/view/MaterializedViewUtils.java  | 14 ++
 1 file changed, 14 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7557bde/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
--
diff --git a/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java 
b/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
index 4dfea75..ea81750 100644
--- a/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
+++ b/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
@@ -26,6 +26,7 @@ import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.dht.Token;
 import org.apache.cassandra.locator.AbstractReplicationStrategy;
+import org.apache.cassandra.service.StorageService;
 import org.apache.cassandra.utils.FBUtilities;
 
 public final class MaterializedViewUtils
@@ -87,8 +88,21 @@ public final class MaterializedViewUtils
 // number of replicas for all of the tokens in the ring.
 assert localBaseEndpoints.size() == localViewEndpoints.size() : 
"Replication strategy should have the same number of endpoints for the base and 
the view";
 int baseIdx = 
localBaseEndpoints.indexOf(FBUtilities.getBroadcastAddress());
+
 if (baseIdx < 0)
+{
+
+if 
(StorageService.instance.getTokenMetadata().pendingEndpointsFor(viewToken, 
keyspaceName).size() > 0)
+{
+//Since there are pending endpoints we are going to store 
hints this in the batchlog regardless.
+//So we can pretend we are the views endpoint.
+
+return FBUtilities.getBroadcastAddress();
+}
+
 throw new RuntimeException("Trying to get the view natural 
endpoint on a non-data replica");
+}
+
 
 return localViewEndpoints.get(baseIdx);
 }

[jira] [Commented] (CASSANDRA-8907) Raise GCInspector alerts to WARN

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730733#comment-14730733
 ] 

Anuj Wadehra commented on CASSANDRA-8907:
-

Hi Joshua McKenzie..I have a different thought on this.. I think this property 
should be enabled by default. Instead, if the property is missing, we should 
take a reasonably high gc warn limit say 1000ms.

I think, no matter what kind of application it is, there is always a gc WARN 
limit for it considering following factors:
1. Application Throughput Requirements/Service Level Agreements  OR
2. phi_convict_threshold : Larger GC Pauses may lead to nodes being marked down 
which is unacceptable by any application. . I think that no application can 
afford unreasonably high GC pauses say >60 secs.

Many Production systems rely on keywords for patrolling logs based on “ERROR” 
and “WARN” and any unreasonably high GC pause should be at WARN level  else 
nodes will be marked down without any warnings.



> Raise GCInspector alerts to WARN
> 
>
> Key: CASSANDRA-8907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8907
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Adam Hattrell
>  Labels: patch
> Attachments: cassnadra-8907.patch
>
>
> I'm fairly regularly running into folks wondering why their applications are 
> reporting down nodes.  Yet, they report, when they grepped the logs they have 
> no WARN or ERRORs listed.
> Nine times out of ten, when I look through the logs we see a ton of ParNew or 
> CMS gc pauses occurring similar to the following:
> INFO [ScheduledTasks:1] 2013-03-07 18:44:46,795 GCInspector.java (line 122) 
> GC for ConcurrentMarkSweep: 1835 ms for 3 collections, 2606015656 used; max 
> is 10611589120
> INFO [ScheduledTasks:1] 2013-03-07 19:45:08,029 GCInspector.java (line 122) 
> GC for ParNew: 9866 ms for 8 collections, 2910124308 used; max is 6358564864
> To my mind these should be WARN's as they have the potential to be 
> significantly impacting the clusters performance as a whole.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730760#comment-14730760
 ] 

Stefan Podkowinski edited comment on CASSANDRA-9590 at 9/4/15 1:07 PM:
---

Most of the failed tests are upgrade tests that will have to clone and build 
another Cassandra version. There's a hardcoded 
{{GIT_REPO="http://git-wip-us.apache.org/repos/asf/cassandra.git"}} value that 
is used in {{ccmlib.repository.clone_development()}}. Unfortunately cloning the 
apache repo sometimes won't work in Ubuntu. Running the tests locally on my 
Ubuntu 14.04 VM will also fail while building the cluster, with only a useful 
message left in the ccm log:

{code}
cat ~/.ccm/repository/last.log
Cloning into bare repository '/home/spod/.ccm/repository/_git_cache_apache'...
fatal: unable to access 
'https://git1-us-west.apache.org/repos/asf/cassandra.git/': GnuTLS recv error 
(-9): A 
TLS packet with unexpected length was received.
{code}

My guess is that the casci Ubuntu VM has the exact same issue with the clone 
process, that's why so many of the upgrade tests failed (see 
[here|http://cassci.datastax.com/job/snazy-9590-3.0-dtest/lastBuild/testReport/upgrade_tests.cql_tests/TestCQL/boolean_test/]
 for example). Maybe you can try to run the tests on a different distro? 

Nonetheless at least the repair tests did reveal an 
[issue|http://cassci.datastax.com/job/snazy-9590-3.0-dtest/lastBuild/testReport/repair_test/TestRepair/dc_repair_test/]
 related to the patch, which I've addressed in a new 
[commit|https://github.com/apache/cassandra/commit/1c1191f0ac7c220e1c28370c7302c438ab98afde].


was (Author: spo...@gmail.com):
Most of the failed tests are upgrade tests that will have to clone and build 
another Cassandra version. There's a hardcoded 
{{GIT_REPO="http://git-wip-us.apache.org/repos/asf/cassandra.git"}} value that 
is used in {{ccmlib.repository.clone_development()}}. Unfortunately cloning the 
apache repo sometimes won't work in Ubuntu. Running the tests locally on my 
Ubuntu 14.04 VM will also fail while building the cluster, with only a useful 
message left in the ccm log:

{code}
cat ~/.ccm/repository/last.log
Cloning into bare repository '/home/spod/.ccm/repository/_git_cache_apache'...
fatal: unable to access 
'https://git1-us-west.apache.org/repos/asf/cassandra.git/': GnuTLS recv error 
(-9): A 
TLS packet with unexpected length was received.
{code}

My guess is that the casci Ubuntu VM has the exact same issue with the clone 
process, that's why so many of the upgrade tests failed. Maybe you can try to 
run the tests on a different distro? 

Nonetheless at least the repair tests did reveal an 
[issue|http://cassci.datastax.com/job/snazy-9590-3.0-dtest/lastBuild/testReport/repair_test/TestRepair/dc_repair_test/]
 related to the patch, which I've addressed in a new 
[commit|https://github.com/apache/cassandra/commit/1c1191f0ac7c220e1c28370c7302c438ab98afde].

> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
> Attachments: nosetest_output.txt
>
>
> Enabling encryption for native transport currently turns SSL exclusively on 
> or off for the opened socket. Migrating from plain to encrypted requires to 
> migrate all native clients as well and redeploy all of them at the same time 
> after starting the SSL enabled Cassandra nodes. 
> This patch would allow to start Cassandra with both an unencrypted and ssl 
> enabled native port. Clients can connect to either, based whether they 
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}} 
> config option. 
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted, 
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set, 
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set, 
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal: 
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
>  [Patch against 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest: 
> 

[1/2] cassandra git commit: Fix return of 'columns()' for AbstractBTreeColumns

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 737b3d0ce -> 721eaf6f0


Fix return of 'columns()' for AbstractBTreeColumns

patch by benedict & slebresne; reviewed by benedict & slebresne for 
CASSANDRA-10220


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/520089b5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/520089b5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/520089b5

Branch: refs/heads/trunk
Commit: 520089b52faef1abbd270c6de130bf6c576ab1c0
Parents: c7557bd
Author: Benedict Elliott Smith 
Authored: Fri Aug 28 14:56:45 2015 +0100
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 15:45:40 2015 +0200

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/PartitionColumns.java   | 14 ++
 .../db/partitions/AbstractBTreePartition.java   | 33 --
 .../db/partitions/AtomicBTreePartition.java |  7 ++-
 .../db/partitions/CachedBTreePartition.java |  5 +--
 .../db/partitions/FilteredPartition.java|  4 +-
 .../db/partitions/ImmutableBTreePartition.java  | 10 ++---
 .../db/partitions/PartitionUpdate.java  | 45 +++-
 8 files changed, 70 insertions(+), 49 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/520089b5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d8a932..e4c5258 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-beta2
+ * Fix columns returned by AbstractBtreePartitions (CASSANDRA-10220)
  * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
  * Fix startup error when upgrading nodes (CASSANDRA-10136)
  * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/520089b5/src/java/org/apache/cassandra/db/PartitionColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/PartitionColumns.java 
b/src/java/org/apache/cassandra/db/PartitionColumns.java
index e1008df..93204f4 100644
--- a/src/java/org/apache/cassandra/db/PartitionColumns.java
+++ b/src/java/org/apache/cassandra/db/PartitionColumns.java
@@ -40,6 +40,7 @@ public class PartitionColumns implements 
Iterable
 
 public PartitionColumns(Columns statics, Columns regulars)
 {
+assert statics != null && regulars != null;
 this.statics = statics;
 this.regulars = regulars;
 }
@@ -61,6 +62,19 @@ public class PartitionColumns implements 
Iterable
 return statics.isEmpty() ? this : new PartitionColumns(Columns.NONE, 
regulars);
 }
 
+public PartitionColumns mergeTo(PartitionColumns that)
+{
+if (this == that)
+return this;
+Columns statics = this.statics.mergeTo(that.statics);
+Columns regulars = this.regulars.mergeTo(that.regulars);
+if (statics == this.statics && regulars == this.regulars)
+return this;
+if (statics == that.statics && regulars == that.regulars)
+return that;
+return new PartitionColumns(statics, regulars);
+}
+
 public boolean isEmpty()
 {
 return statics.isEmpty() && regulars.isEmpty();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/520089b5/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--
diff --git 
a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java 
b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
index 41015b0..da20c91 100644
--- a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
+++ b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
@@ -32,31 +32,32 @@ import static 
org.apache.cassandra.utils.btree.BTree.Dir.desc;
 
 public abstract class AbstractBTreePartition implements Partition, 
Iterable
 {
-protected static final Holder EMPTY = new Holder(BTree.empty(), 
DeletionInfo.LIVE, Rows.EMPTY_STATIC_ROW, EncodingStats.NO_STATS);
+protected static final Holder EMPTY = new Holder(PartitionColumns.NONE, 
BTree.empty(), DeletionInfo.LIVE, Rows.EMPTY_STATIC_ROW, 
EncodingStats.NO_STATS);
 
 protected final CFMetaData metadata;
 protected final DecoratedKey partitionKey;
-protected final PartitionColumns columns;
+
 protected abstract Holder holder();
 protected abstract boolean canHaveShadowedData();
 
-protected AbstractBTreePartition(CFMetaData metadata, DecoratedKey 
partitionKey, PartitionColumns columns)
+protected AbstractBTreePartition(CFMetaData metadata, 

[jira] [Updated] (CASSANDRA-8844) Change Data Capture (CDC)

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8844:
--
Assignee: Joshua McKenzie

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
> Fix For: 3.x
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order to make consuming a change log easy and efficient to do with low 
> latency, the following could supplement the approach outlined above
> - Instead of writing to a logfile, by default, Cassandra could expose a 
> socket for a daemon to connect to, and from which it could pull each row.
> - Cassandra would have a limited buffer for 

[jira] [Updated] (CASSANDRA-8844) Change Data Capture (CDC)

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8844:
--
Priority: Critical  (was: Major)

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.x
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order to make consuming a change log easy and efficient to do with low 
> latency, the following could supplement the approach outlined above
> - Instead of writing to a logfile, by default, Cassandra could expose a 
> socket for a daemon to connect to, and from which it could pull each row.
> - 

[jira] [Commented] (CASSANDRA-8907) Raise GCInspector alerts to WARN

[Anuj Wadehra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730762#comment-14730762
 ] 

Anuj Wadehra commented on CASSANDRA-8907:
-

Yes. I Agree. We can negotiate on a default value. But my main concern is not 
the default value. I am suggesting that this property should be enabled by 
default as it has severe implications of marking nodes as DOWN, irrespective of 
the type of application and it's sensitivity to GC pauses. 

> Raise GCInspector alerts to WARN
> 
>
> Key: CASSANDRA-8907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8907
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Adam Hattrell
>  Labels: patch
> Attachments: cassnadra-8907.patch
>
>
> I'm fairly regularly running into folks wondering why their applications are 
> reporting down nodes.  Yet, they report, when they grepped the logs they have 
> no WARN or ERRORs listed.
> Nine times out of ten, when I look through the logs we see a ton of ParNew or 
> CMS gc pauses occurring similar to the following:
> INFO [ScheduledTasks:1] 2013-03-07 18:44:46,795 GCInspector.java (line 122) 
> GC for ConcurrentMarkSweep: 1835 ms for 3 collections, 2606015656 used; max 
> is 10611589120
> INFO [ScheduledTasks:1] 2013-03-07 19:45:08,029 GCInspector.java (line 122) 
> GC for ParNew: 9866 ms for 8 collections, 2910124308 used; max is 6358564864
> To my mind these should be WARN's as they have the potential to be 
> significantly impacting the clusters performance as a whole.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730760#comment-14730760
 ] 

Stefan Podkowinski edited comment on CASSANDRA-9590 at 9/4/15 1:05 PM:
---

Most of the failed tests are upgrade tests that will have to clone and build 
another Cassandra version. There's a hardcoded 
{{GIT_REPO="http://git-wip-us.apache.org/repos/asf/cassandra.git"}} value that 
is used in {{ccmlib.repository.clone_development()}}. Unfortunately cloning the 
apache repo sometimes won't work in Ubuntu. Running the tests locally on my 
Ubuntu 14.04 VM will also fail while building the cluster, with only a useful 
message left in the ccm log:

{code}
cat ~/.ccm/repository/last.log
Cloning into bare repository '/home/spod/.ccm/repository/_git_cache_apache'...
fatal: unable to access 
'https://git1-us-west.apache.org/repos/asf/cassandra.git/': GnuTLS recv error 
(-9): A 
TLS packet with unexpected length was received.
{code}

My guess is that the casci Ubuntu VM has the exact same issue with the clone 
process, that's why so many of the upgrade tests failed. Maybe you can try to 
run the tests on a different distro? 

Nonetheless at least the repair tests did reveal an 
[issue|http://cassci.datastax.com/job/snazy-9590-3.0-dtest/lastBuild/testReport/repair_test/TestRepair/dc_repair_test/]
 related to the patch, which I've addressed in a new 
[commit|https://github.com/apache/cassandra/commit/1c1191f0ac7c220e1c28370c7302c438ab98afde].


was (Author: spo...@gmail.com):
Most of the failed tests are upgrade tests that will have to clone and build 
another Cassandra version. There's a hardcoded 
{{GIT_REPO="http://git-wip-us.apache.org/repos/asf/cassandra.git"}} value that 
is used in {{ccmlib.repository.clone_development()}}. Unfortunately cloning the 
apache repo sometimes won't work in Ubuntu. Running the tests locally on my 
Ubuntu 14.04 VM will also fail while building the cluster, with only a useful 
message left in the ccm log:

{code}
cat ~/.ccm/repository/last.log
Cloning into bare repository '/home/spod/.ccm/repository/_git_cache_apache'...
fatal: unable to access 
'https://git1-us-west.apache.org/repos/asf/cassandra.git/': GnuTLS recv error 
(-9): A 
TLS packet with unexpected length was received.
{code}

My guess is that the casci Ubuntu VM has the exact same issue with the clone 
process, that's why so many of the integration tests failed. Maybe you can try 
to run the tests on a different distro? 

Nonetheless at least the repair tests did reveal an 
[issue|http://cassci.datastax.com/job/snazy-9590-3.0-dtest/lastBuild/testReport/repair_test/TestRepair/dc_repair_test/]
 related to the patch, which I've addressed in a new 
[commit|https://github.com/apache/cassandra/commit/1c1191f0ac7c220e1c28370c7302c438ab98afde].

> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
> Attachments: nosetest_output.txt
>
>
> Enabling encryption for native transport currently turns SSL exclusively on 
> or off for the opened socket. Migrating from plain to encrypted requires to 
> migrate all native clients as well and redeploy all of them at the same time 
> after starting the SSL enabled Cassandra nodes. 
> This patch would allow to start Cassandra with both an unencrypted and ssl 
> enabled native port. Clients can connect to either, based whether they 
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}} 
> config option. 
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted, 
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set, 
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set, 
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal: 
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
>  [Patch against 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest: 
> [Branch|https://github.com/spodkowinski/cassandra-dtest/tree/cassandra-9590], 
> [Diff 
> master|https://github.com/riptano/cassandra-dtest/compare/master...spodkowinski:cassandra-9590],
> 

[jira] [Commented] (CASSANDRA-10146) Deprecate v1 and v2 protocol in 2.2, drop support in 3.0

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730864#comment-14730864
 ] 

Benjamin Lerer commented on CASSANDRA-10146:


The patch for 2.1 is 
[here|https://github.com/apache/cassandra/compare/trunk...blerer:10146-2.1]
The patch for 2.2 is 
[here|https://github.com/apache/cassandra/compare/trunk...blerer:10146-2.2]
The patch for 3.0 is 
[here|https://github.com/apache/cassandra/compare/trunk...blerer:10146-3.0]

* the unit test results for 2.1 are 
[here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-2.1-testall/2/testReport/]
* the dtest results for 2.1 are 
[here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10127-dtest/5/]
* the unit test results for 2.2 are 
[here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-2.2-testall/2/]
* the dtest results for 2.2 are 
[here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-2.2-dtest/3/]
* the unit test results for 3.0 are 
[here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-3.0-testall/2/]
* the dtest results for 3.0 are 
[here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-3.0-dtest/5/]

> Deprecate v1 and v2 protocol in 2.2, drop support in 3.0
> 
>
> Key: CASSANDRA-10146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10146
> Project: Cassandra
>  Issue Type: Task
>  Components: Core
>Reporter: Tyler Hobbs
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 2.2.x, 3.0.0 rc1
>
>
> In 3.0, we would like to use frozen collections in the system keyspaces, and 
> it seems likely that we will eventually want to use tuples or nested 
> collections as well.  Drivers that only support protocol versions 1 and 2 
> will not be able to read these system keyspaces because they cannot decode 
> those types.
> I think this is a good time to start deprecating and dropping support for old 
> protocol versions.  The v3 protocol was introduced in 2.1, so if we remove 
> support for v1 and v2 in 3.0, that gives users two major versions to upgrade 
> their drivers.  Fortunately, upgrading drivers to a version that supports the 
> v3 protocol is generally straightforward.
> The benefits of doing this are:
> * We can use new types in the system keyspaces
> * We can eliminate protocol-version-specific encoding and decoding of 
> collections within Cassandra
> * Driver maintainers can eventually drop support for old protocol versions 
> once all C* versions that support them are EOL
> To avoid a hard drop of v1 and v2 support in 3.0, I propose that we also 
> officially deprecate these in 2.2.  Unfortunately, we don't have 
> protocol-level warnings until v4, so we can't use that to notify users of the 
> deprecation, but the combination of a NEWS.txt entry, warning logs, and 
> (potentially) driver-level warnings should suffice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10262) Bootstrapping nodes failing to apply view updates correctly.

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-10262:
---
Fix Version/s: (was: 3.0.0 rc1)
   3.0 beta 2

> Bootstrapping nodes failing to apply view updates correctly.
> 
>
> Key: CASSANDRA-10262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10262
> Project: Cassandra
>  Issue Type: Bug
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.0 beta 2
>
>
> The bootstrapping dtests started failing after CASSANDRA-10164
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/130/testReport/junit/materialized_views_test/TestMaterializedViews/add_node_after_mv_test/
> The underlying issue is we were never applying MV updates to non-joined nodes 
> before. Now that we are we can see bootstrapping nodes need to handle pending 
> nodes when calculating the paired replica in the case there is no natural 
> endpoints for that node (since it is pending).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Alan Boudreault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Boudreault updated CASSANDRA-9664:
---
Tester: Alan Boudreault

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730760#comment-14730760
 ] 

Stefan Podkowinski commented on CASSANDRA-9590:
---

Most of the failed tests are upgrade tests that will have to clone and build 
another Cassandra version. There's a hardcoded 
{{GIT_REPO="http://git-wip-us.apache.org/repos/asf/cassandra.git"}} value that 
is used in {{ccmlib.repository.clone_development()}}. Unfortunately cloning the 
apache repo sometimes won't work in Ubuntu. Running the tests locally on my 
Ubuntu 14.04 VM will also fail while building the cluster, with only a useful 
message left in the ccm log:

{code}
cat ~/.ccm/repository/last.log
Cloning into bare repository '/home/spod/.ccm/repository/_git_cache_apache'...
fatal: unable to access 
'https://git1-us-west.apache.org/repos/asf/cassandra.git/': GnuTLS recv error 
(-9): A 
TLS packet with unexpected length was received.
{code}

My guess is that the casci Ubuntu VM has the exact same issue with the clone 
process, that's why so many of the integration tests failed. Maybe you can try 
to run the tests on a different distro? 

Nonetheless at least the repair tests did reveal an 
[issue|http://cassci.datastax.com/job/snazy-9590-3.0-dtest/lastBuild/testReport/repair_test/TestRepair/dc_repair_test/]
 related to the patch, which I've addressed in a new 
[commit|https://github.com/apache/cassandra/commit/1c1191f0ac7c220e1c28370c7302c438ab98afde].

> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
> Attachments: nosetest_output.txt
>
>
> Enabling encryption for native transport currently turns SSL exclusively on 
> or off for the opened socket. Migrating from plain to encrypted requires to 
> migrate all native clients as well and redeploy all of them at the same time 
> after starting the SSL enabled Cassandra nodes. 
> This patch would allow to start Cassandra with both an unencrypted and ssl 
> enabled native port. Clients can connect to either, based whether they 
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}} 
> config option. 
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted, 
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set, 
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set, 
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal: 
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
>  [Patch against 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest: 
> [Branch|https://github.com/spodkowinski/cassandra-dtest/tree/cassandra-9590], 
> [Diff 
> master|https://github.com/riptano/cassandra-dtest/compare/master...spodkowinski:cassandra-9590],
>  [Pull Request|https://github.com/riptano/cassandra-dtest/pull/530]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730823#comment-14730823
 ] 

Robert Stupp commented on CASSANDRA-9590:
-

Ok, I've reset "my" branch to your branch and rebased against current 3.0 
branch. Let's see what cassci says.

> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
> Attachments: nosetest_output.txt
>
>
> Enabling encryption for native transport currently turns SSL exclusively on 
> or off for the opened socket. Migrating from plain to encrypted requires to 
> migrate all native clients as well and redeploy all of them at the same time 
> after starting the SSL enabled Cassandra nodes. 
> This patch would allow to start Cassandra with both an unencrypted and ssl 
> enabled native port. Clients can connect to either, based whether they 
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}} 
> config option. 
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted, 
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set, 
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set, 
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal: 
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
>  [Patch against 
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest: 
> [Branch|https://github.com/spodkowinski/cassandra-dtest/tree/cassandra-9590], 
> [Diff 
> master|https://github.com/riptano/cassandra-dtest/compare/master...spodkowinski:cassandra-9590],
>  [Pull Request|https://github.com/riptano/cassandra-dtest/pull/530]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[slebresne]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/721eaf6f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/721eaf6f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/721eaf6f

Branch: refs/heads/trunk
Commit: 721eaf6f0eabadb77da27a8765c7884415a1337b
Parents: 737b3d0 520089b
Author: Sylvain Lebresne 
Authored: Fri Sep 4 15:46:55 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 15:46:55 2015 +0200

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/PartitionColumns.java   | 14 ++
 .../db/partitions/AbstractBTreePartition.java   | 33 --
 .../db/partitions/AtomicBTreePartition.java |  7 ++-
 .../db/partitions/CachedBTreePartition.java |  5 +--
 .../db/partitions/FilteredPartition.java|  4 +-
 .../db/partitions/ImmutableBTreePartition.java  | 10 ++---
 .../db/partitions/PartitionUpdate.java  | 45 +++-
 8 files changed, 70 insertions(+), 49 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/721eaf6f/CHANGES.txt
--
diff --cc CHANGES.txt
index 84eed2a,e4c5258..e726ada
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,5 +1,9 @@@
 +3.2
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0.0-beta2
+  * Fix columns returned by AbstractBtreePartitions (CASSANDRA-10220)
   * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
   * Fix startup error when upgrading nodes (CASSANDRA-10136)
   * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)

cassandra git commit: Fix return of 'columns()' for AbstractBTreeColumns

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 ab962202a -> c3bc85641


Fix return of 'columns()' for AbstractBTreeColumns

patch by benedict & slebresne; reviewed by benedict & slebresne for 
CASSANDRA-10220


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c3bc8564
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c3bc8564
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c3bc8564

Branch: refs/heads/cassandra-3.0
Commit: c3bc85641bc3297d151cdd983666544dd240f941
Parents: ab96220
Author: Benedict Elliott Smith 
Authored: Fri Aug 28 14:56:45 2015 +0100
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 16:18:50 2015 +0200

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/PartitionColumns.java   | 14 ++
 .../db/partitions/AbstractBTreePartition.java   | 33 --
 .../db/partitions/AtomicBTreePartition.java |  7 ++-
 .../db/partitions/CachedBTreePartition.java |  5 +--
 .../db/partitions/FilteredPartition.java|  4 +-
 .../db/partitions/ImmutableBTreePartition.java  | 10 ++---
 .../db/partitions/PartitionUpdate.java  | 45 +++-
 8 files changed, 70 insertions(+), 49 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c3bc8564/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 021d90c..d4e6771 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-beta2
+ * Fix columns returned by AbstractBtreePartitions (CASSANDRA-10220)
  * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
  * Fix startup error when upgrading nodes (CASSANDRA-10136)
  * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c3bc8564/src/java/org/apache/cassandra/db/PartitionColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/PartitionColumns.java 
b/src/java/org/apache/cassandra/db/PartitionColumns.java
index e1008df..93204f4 100644
--- a/src/java/org/apache/cassandra/db/PartitionColumns.java
+++ b/src/java/org/apache/cassandra/db/PartitionColumns.java
@@ -40,6 +40,7 @@ public class PartitionColumns implements 
Iterable
 
 public PartitionColumns(Columns statics, Columns regulars)
 {
+assert statics != null && regulars != null;
 this.statics = statics;
 this.regulars = regulars;
 }
@@ -61,6 +62,19 @@ public class PartitionColumns implements 
Iterable
 return statics.isEmpty() ? this : new PartitionColumns(Columns.NONE, 
regulars);
 }
 
+public PartitionColumns mergeTo(PartitionColumns that)
+{
+if (this == that)
+return this;
+Columns statics = this.statics.mergeTo(that.statics);
+Columns regulars = this.regulars.mergeTo(that.regulars);
+if (statics == this.statics && regulars == this.regulars)
+return this;
+if (statics == that.statics && regulars == that.regulars)
+return that;
+return new PartitionColumns(statics, regulars);
+}
+
 public boolean isEmpty()
 {
 return statics.isEmpty() && regulars.isEmpty();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c3bc8564/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--
diff --git 
a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java 
b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
index 41015b0..da20c91 100644
--- a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
+++ b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
@@ -32,31 +32,32 @@ import static 
org.apache.cassandra.utils.btree.BTree.Dir.desc;
 
 public abstract class AbstractBTreePartition implements Partition, 
Iterable
 {
-protected static final Holder EMPTY = new Holder(BTree.empty(), 
DeletionInfo.LIVE, Rows.EMPTY_STATIC_ROW, EncodingStats.NO_STATS);
+protected static final Holder EMPTY = new Holder(PartitionColumns.NONE, 
BTree.empty(), DeletionInfo.LIVE, Rows.EMPTY_STATIC_ROW, 
EncodingStats.NO_STATS);
 
 protected final CFMetaData metadata;
 protected final DecoratedKey partitionKey;
-protected final PartitionColumns columns;
+
 protected abstract Holder holder();
 protected abstract boolean canHaveShadowedData();
 
-protected AbstractBTreePartition(CFMetaData metadata, DecoratedKey 
partitionKey, PartitionColumns columns)
+protected 

[21/22] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jbellis]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab962202
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab962202
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab962202

Branch: refs/heads/trunk
Commit: ab962202ae4b1f127441dcf8e733b8d2d73ddf7e
Parents: c7557bd 25373e9
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:18:04 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:18:04 2015 -0500

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/StorageService.java  | 2 ++
 src/java/org/apache/cassandra/tools/nodetool/Decommission.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab962202/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab962202/src/java/org/apache/cassandra/service/StorageService.java
--

[jira] [Updated] (CASSANDRA-9738) Migrate key-cache to be fully off-heap

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-9738:
--
Fix Version/s: (was: 3.0 beta 2)
   3.0.0 rc1

> Migrate key-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and 
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure 
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some 
> benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10161) Composite case-sensitive primary key: first item is not quoted in DESCRIBE TABLE

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730903#comment-14730903
 ] 

Adam Holmberg commented on CASSANDRA-10161:
---

Thanks for the ping. I'll be rebasing the 3.0 stuff onto master today. Will 
note here when it's ready to roll.

> Composite case-sensitive primary key: first item is not quoted in DESCRIBE 
> TABLE
> 
>
> Key: CASSANDRA-10161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10161
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Julien Moreau
>Assignee: Carl Yeksigian
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.1.x
>
>
> A table is created with case-sensitive composite primary key:
> {code}
> CREATE TABLE foo (
>  "Key1" text,
>  "Key2" text,
>  "Value" text
>  PRIMARY KEY ("Key1", "Key2")
> );
> {code}
> In the result of {{DESCRIBE TABLE foo;}}:
> {code}
> CREATE TABLE foo (
>  "Key1" text,
>  "Key2" text,
>  "Value" text
>  PRIMARY KEY (Key1, "Key2")
> );
> {code}
> {{Key1}} is not quoted.
> When trying to re-create the table with this description, there is an error:
> {code}
> InvalidRequest: code=2200 [Invalid query] message="Unknown definition key1 
> referenced in PRIMARY KEY"
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8907) Raise GCInspector alerts to WARN

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730751#comment-14730751
 ] 

Joshua McKenzie commented on CASSANDRA-8907:


bq. any unreasonably high GC
While I *personally * agree that 1000ms is unreasonably high, I'd also say that 
I think 500ms is unreasonably high. Or 250ms. Unfortunately there's no single 
right answer for a default value for GC duration on a node and I was thinking 
we err on the side of Least Astonishment.

That being said I'm not married to the idea and I'm core dev, not in ops. For 
the folks in ops reading this - what are your thoughts on a default at 1k vs. 
disabled? cc [~jeromatron] / [~johnny15676] / [~ahattrell]

> Raise GCInspector alerts to WARN
> 
>
> Key: CASSANDRA-8907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8907
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Adam Hattrell
>  Labels: patch
> Attachments: cassnadra-8907.patch
>
>
> I'm fairly regularly running into folks wondering why their applications are 
> reporting down nodes.  Yet, they report, when they grepped the logs they have 
> no WARN or ERRORs listed.
> Nine times out of ten, when I look through the logs we see a ton of ParNew or 
> CMS gc pauses occurring similar to the following:
> INFO [ScheduledTasks:1] 2013-03-07 18:44:46,795 GCInspector.java (line 122) 
> GC for ConcurrentMarkSweep: 1835 ms for 3 collections, 2606015656 used; max 
> is 10611589120
> INFO [ScheduledTasks:1] 2013-03-07 19:45:08,029 GCInspector.java (line 122) 
> GC for ParNew: 9866 ms for 8 collections, 2910124308 used; max is 6358564864
> To my mind these should be WARN's as they have the potential to be 
> significantly impacting the clusters performance as a whole.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jake]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/737b3d0c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/737b3d0c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/737b3d0c

Branch: refs/heads/trunk
Commit: 737b3d0cec1236142c8e8fce0450001a3ebab258
Parents: ae9914b c7557bd
Author: T Jake Luciani 
Authored: Fri Sep 4 09:11:30 2015 -0400
Committer: T Jake Luciani 
Committed: Fri Sep 4 09:11:30 2015 -0400

--
 .../cassandra/db/view/MaterializedViewUtils.java  | 14 ++
 1 file changed, 14 insertions(+)
--

[1/2] cassandra git commit: fix

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/trunk ae9914bb5 -> 737b3d0ce


fix


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7557bde
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7557bde
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7557bde

Branch: refs/heads/trunk
Commit: c7557bdecda6ffe76bb5ee62be4c2d6c53804681
Parents: a22db5b
Author: T Jake Luciani 
Authored: Thu Sep 3 14:53:00 2015 -0400
Committer: T Jake Luciani 
Committed: Fri Sep 4 09:10:49 2015 -0400

--
 .../cassandra/db/view/MaterializedViewUtils.java  | 14 ++
 1 file changed, 14 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7557bde/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
--
diff --git a/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java 
b/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
index 4dfea75..ea81750 100644
--- a/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
+++ b/src/java/org/apache/cassandra/db/view/MaterializedViewUtils.java
@@ -26,6 +26,7 @@ import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.dht.Token;
 import org.apache.cassandra.locator.AbstractReplicationStrategy;
+import org.apache.cassandra.service.StorageService;
 import org.apache.cassandra.utils.FBUtilities;
 
 public final class MaterializedViewUtils
@@ -87,8 +88,21 @@ public final class MaterializedViewUtils
 // number of replicas for all of the tokens in the ring.
 assert localBaseEndpoints.size() == localViewEndpoints.size() : 
"Replication strategy should have the same number of endpoints for the base and 
the view";
 int baseIdx = 
localBaseEndpoints.indexOf(FBUtilities.getBroadcastAddress());
+
 if (baseIdx < 0)
+{
+
+if 
(StorageService.instance.getTokenMetadata().pendingEndpointsFor(viewToken, 
keyspaceName).size() > 0)
+{
+//Since there are pending endpoints we are going to store 
hints this in the batchlog regardless.
+//So we can pretend we are the views endpoint.
+
+return FBUtilities.getBroadcastAddress();
+}
+
 throw new RuntimeException("Trying to get the view natural 
endpoint on a non-data replica");
+}
+
 
 return localViewEndpoints.get(baseIdx);
 }

[jira] [Resolved] (CASSANDRA-10220) Memtables do not handle column changes for their PartitionColumns

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-10220.
--
Resolution: Fixed

Alright, cassci seems to be happy (there was one timeout during the unit test, 
but that test passed multiple time locally, though is a somewhat long test (for 
a unit test) so pretty sure this was just cassci being slow. ).

> Memtables do not handle column changes for their PartitionColumns
> -
>
> Key: CASSANDRA-10220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10220
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> There are a couple of inherited issues around the {{PartitionColumns}} in 
> {{AbstractBTreePartition}}. Firstly, we do not always return these columns, 
> but sometimes the metadata columns. Secondly, the columns cannot be updated 
> with new columns, and the metadata columns will not show dropped columns. 
> This should most likely be moved into the {{Holder}}, and updated along with 
> any other updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[14/22] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jbellis]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3f505191
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3f505191
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3f505191

Branch: refs/heads/cassandra-2.2
Commit: 3f50519141df4d0b4c5224e2652c96cfa0333feb
Parents: 1f13528 5ec0204
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:26 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:26 2015 -0500

--

--

[09/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27754c0a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27754c0a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27754c0a

Branch: refs/heads/cassandra-2.2
Commit: 27754c0a41087a2419c09fd99707fd7235cce4ad
Parents: fe80052
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:15:56 2015 -0500

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeTool.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 681a40b..6d43c98 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,8 @@
  * Only check KeyCache when it is enabled
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.1.9

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 56ddd63..acc82e6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3281,6 +3281,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 8e88f45..277f42c 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1471,6 +1471,9 @@ public class NodeTool
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }

[02/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bc56c3e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bc56c3e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bc56c3e

Branch: refs/heads/cassandra-3.0
Commit: 5bc56c3e407f9deb0c931f63b5d6e4c768a0632d
Parents: 517058f
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:13:53 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeCmd.java  | 9 -
 3 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0e4ade3..d4cc15f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.17
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
  * Backport CASSANDRA-8013 to 2.0 (CASSANDRA-10144)
  * Make getFullyExpiredSSTables less expensive (CASSANDRA-9882)
  * Add tool to find why expired sstables are not getting dropped 
(CASSANDRA-10015)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 745fe4c..5ac4980 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2963,6 +2963,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index b006d69..8071013 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -1366,7 +1366,8 @@ public class NodeCmd
 System.err.println("Decommission will decommission the 
node you are connected to and does not take arguments!");
 System.exit(1);
 }
-probe.decommission();
+try { probe.decommission(); }
+catch (UnsupportedOperationException e) { err("Unsupported 
operation: " + e.getMessage()); }
 break;
 
 case DRAIN :
@@ -1678,6 +1679,12 @@ public class NodeCmd
 System.exit(1);
 }
 
+private static void err(String useStr)
+{
+System.err.println(useStr);
+System.exit(1);
+}
+
 private static void err(Exception e, String errStr)
 {
 System.err.println(errStr);

[01/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 517058feb -> 5bc56c3e4
  refs/heads/cassandra-2.1 fe8005257 -> 5ec020490
  refs/heads/cassandra-2.2 1f1352821 -> 25373e994
  refs/heads/cassandra-3.0 c7557bdec -> ab962202a
  refs/heads/trunk 721eaf6f0 -> 5bb80362d


refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bc56c3e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bc56c3e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bc56c3e

Branch: refs/heads/cassandra-2.0
Commit: 5bc56c3e407f9deb0c931f63b5d6e4c768a0632d
Parents: 517058f
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:13:53 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeCmd.java  | 9 -
 3 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0e4ade3..d4cc15f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.17
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
  * Backport CASSANDRA-8013 to 2.0 (CASSANDRA-10144)
  * Make getFullyExpiredSSTables less expensive (CASSANDRA-9882)
  * Add tool to find why expired sstables are not getting dropped 
(CASSANDRA-10015)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 745fe4c..5ac4980 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2963,6 +2963,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index b006d69..8071013 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -1366,7 +1366,8 @@ public class NodeCmd
 System.err.println("Decommission will decommission the 
node you are connected to and does not take arguments!");
 System.exit(1);
 }
-probe.decommission();
+try { probe.decommission(); }
+catch (UnsupportedOperationException e) { err("Unsupported 
operation: " + e.getMessage()); }
 break;
 
 case DRAIN :
@@ -1678,6 +1679,12 @@ public class NodeCmd
 System.exit(1);
 }
 
+private static void err(String useStr)
+{
+System.err.println(useStr);
+System.exit(1);
+}
+
 private static void err(Exception e, String errStr)
 {
 System.err.println(errStr);

[18/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25373e99
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25373e99
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25373e99

Branch: refs/heads/cassandra-2.2
Commit: 25373e9945a3de7a7caf76281098e44bf8c6a8a0
Parents: 3f50519
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:50 2015 -0500

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/StorageService.java  | 2 ++
 src/java/org/apache/cassandra/tools/nodetool/Decommission.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20e1297..7ac7b23 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -10,6 +10,8 @@ Merged from 2.1:
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
  * Add nodetool gettraceprobability command (CASSANDRA-10234)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.2.1

http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 883c96b..ef795f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3294,6 +3294,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/Decommission.java 
b/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
index a3feb67..34890e0 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
@@ -34,6 +34,9 @@ public class Decommission extends NodeToolCmd
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }
\ No newline at end of file

[10/22] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[jbellis]

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ec02049
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ec02049
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ec02049

Branch: refs/heads/cassandra-2.2
Commit: 5ec020490f46679a3f13cb96b2f7dfb56744dea2
Parents: 27754c0 5bc56c3
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:16 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:16 2015 -0500

--

--

[03/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bc56c3e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bc56c3e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bc56c3e

Branch: refs/heads/cassandra-2.1
Commit: 5bc56c3e407f9deb0c931f63b5d6e4c768a0632d
Parents: 517058f
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:13:53 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeCmd.java  | 9 -
 3 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0e4ade3..d4cc15f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.17
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
  * Backport CASSANDRA-8013 to 2.0 (CASSANDRA-10144)
  * Make getFullyExpiredSSTables less expensive (CASSANDRA-9882)
  * Add tool to find why expired sstables are not getting dropped 
(CASSANDRA-10015)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 745fe4c..5ac4980 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2963,6 +2963,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index b006d69..8071013 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -1366,7 +1366,8 @@ public class NodeCmd
 System.err.println("Decommission will decommission the 
node you are connected to and does not take arguments!");
 System.exit(1);
 }
-probe.decommission();
+try { probe.decommission(); }
+catch (UnsupportedOperationException e) { err("Unsupported 
operation: " + e.getMessage()); }
 break;
 
 case DRAIN :
@@ -1678,6 +1679,12 @@ public class NodeCmd
 System.exit(1);
 }
 
+private static void err(String useStr)
+{
+System.err.println(useStr);
+System.exit(1);
+}
+
 private static void err(Exception e, String errStr)
 {
 System.err.println(errStr);

[20/22] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jbellis]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab962202
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab962202
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab962202

Branch: refs/heads/cassandra-3.0
Commit: ab962202ae4b1f127441dcf8e733b8d2d73ddf7e
Parents: c7557bd 25373e9
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:18:04 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:18:04 2015 -0500

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/StorageService.java  | 2 ++
 src/java/org/apache/cassandra/tools/nodetool/Decommission.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab962202/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab962202/src/java/org/apache/cassandra/service/StorageService.java
--

[17/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25373e99
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25373e99
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25373e99

Branch: refs/heads/cassandra-3.0
Commit: 25373e9945a3de7a7caf76281098e44bf8c6a8a0
Parents: 3f50519
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:50 2015 -0500

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/StorageService.java  | 2 ++
 src/java/org/apache/cassandra/tools/nodetool/Decommission.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20e1297..7ac7b23 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -10,6 +10,8 @@ Merged from 2.1:
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
  * Add nodetool gettraceprobability command (CASSANDRA-10234)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.2.1

http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 883c96b..ef795f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3294,6 +3294,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/Decommission.java 
b/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
index a3feb67..34890e0 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
@@ -34,6 +34,9 @@ public class Decommission extends NodeToolCmd
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }
\ No newline at end of file

[04/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bc56c3e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bc56c3e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bc56c3e

Branch: refs/heads/trunk
Commit: 5bc56c3e407f9deb0c931f63b5d6e4c768a0632d
Parents: 517058f
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:13:53 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeCmd.java  | 9 -
 3 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0e4ade3..d4cc15f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.17
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
  * Backport CASSANDRA-8013 to 2.0 (CASSANDRA-10144)
  * Make getFullyExpiredSSTables less expensive (CASSANDRA-9882)
  * Add tool to find why expired sstables are not getting dropped 
(CASSANDRA-10015)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 745fe4c..5ac4980 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2963,6 +2963,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index b006d69..8071013 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -1366,7 +1366,8 @@ public class NodeCmd
 System.err.println("Decommission will decommission the 
node you are connected to and does not take arguments!");
 System.exit(1);
 }
-probe.decommission();
+try { probe.decommission(); }
+catch (UnsupportedOperationException e) { err("Unsupported 
operation: " + e.getMessage()); }
 break;
 
 case DRAIN :
@@ -1678,6 +1679,12 @@ public class NodeCmd
 System.exit(1);
 }
 
+private static void err(String useStr)
+{
+System.err.println(useStr);
+System.exit(1);
+}
+
 private static void err(Exception e, String errStr)
 {
 System.err.println(errStr);

[19/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25373e99
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25373e99
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25373e99

Branch: refs/heads/trunk
Commit: 25373e9945a3de7a7caf76281098e44bf8c6a8a0
Parents: 3f50519
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:50 2015 -0500

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/StorageService.java  | 2 ++
 src/java/org/apache/cassandra/tools/nodetool/Decommission.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20e1297..7ac7b23 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -10,6 +10,8 @@ Merged from 2.1:
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
  * Add nodetool gettraceprobability command (CASSANDRA-10234)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.2.1

http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 883c96b..ef795f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3294,6 +3294,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/25373e99/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/Decommission.java 
b/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
index a3feb67..34890e0 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Decommission.java
@@ -34,6 +34,9 @@ public class Decommission extends NodeToolCmd
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }
\ No newline at end of file

[05/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bc56c3e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bc56c3e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bc56c3e

Branch: refs/heads/cassandra-2.2
Commit: 5bc56c3e407f9deb0c931f63b5d6e4c768a0632d
Parents: 517058f
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:13:53 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeCmd.java  | 9 -
 3 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0e4ade3..d4cc15f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.17
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
  * Backport CASSANDRA-8013 to 2.0 (CASSANDRA-10144)
  * Make getFullyExpiredSSTables less expensive (CASSANDRA-9882)
  * Add tool to find why expired sstables are not getting dropped 
(CASSANDRA-10015)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 745fe4c..5ac4980 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2963,6 +2963,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bc56c3e/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index b006d69..8071013 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -1366,7 +1366,8 @@ public class NodeCmd
 System.err.println("Decommission will decommission the 
node you are connected to and does not take arguments!");
 System.exit(1);
 }
-probe.decommission();
+try { probe.decommission(); }
+catch (UnsupportedOperationException e) { err("Unsupported 
operation: " + e.getMessage()); }
 break;
 
 case DRAIN :
@@ -1678,6 +1679,12 @@ public class NodeCmd
 System.exit(1);
 }
 
+private static void err(String useStr)
+{
+System.err.println(useStr);
+System.exit(1);
+}
+
 private static void err(Exception e, String errStr)
 {
 System.err.println(errStr);

[22/22] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jbellis]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bb80362
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bb80362
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bb80362

Branch: refs/heads/trunk
Commit: 5bb80362d2d0533884f5a3af8892e6ff62d0bfff
Parents: 721eaf6 ab96220
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:18:14 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:18:14 2015 -0500

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/StorageService.java  | 2 ++
 src/java/org/apache/cassandra/tools/nodetool/Decommission.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bb80362/CHANGES.txt
--

[06/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27754c0a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27754c0a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27754c0a

Branch: refs/heads/cassandra-2.1
Commit: 27754c0a41087a2419c09fd99707fd7235cce4ad
Parents: fe80052
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:15:56 2015 -0500

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeTool.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 681a40b..6d43c98 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,8 @@
  * Only check KeyCache when it is enabled
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.1.9

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 56ddd63..acc82e6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3281,6 +3281,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 8e88f45..277f42c 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1471,6 +1471,9 @@ public class NodeTool
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }

[08/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27754c0a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27754c0a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27754c0a

Branch: refs/heads/trunk
Commit: 27754c0a41087a2419c09fd99707fd7235cce4ad
Parents: fe80052
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:15:56 2015 -0500

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeTool.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 681a40b..6d43c98 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,8 @@
  * Only check KeyCache when it is enabled
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.1.9

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 56ddd63..acc82e6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3281,6 +3281,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 8e88f45..277f42c 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1471,6 +1471,9 @@ public class NodeTool
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }

[16/22] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jbellis]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3f505191
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3f505191
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3f505191

Branch: refs/heads/trunk
Commit: 3f50519141df4d0b4c5224e2652c96cfa0333feb
Parents: 1f13528 5ec0204
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:26 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:26 2015 -0500

--

--

[13/22] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[jbellis]

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ec02049
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ec02049
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ec02049

Branch: refs/heads/cassandra-2.1
Commit: 5ec020490f46679a3f13cb96b2f7dfb56744dea2
Parents: 27754c0 5bc56c3
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:16 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:16 2015 -0500

--

--

[12/22] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[jbellis]

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ec02049
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ec02049
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ec02049

Branch: refs/heads/trunk
Commit: 5ec020490f46679a3f13cb96b2f7dfb56744dea2
Parents: 27754c0 5bc56c3
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:16 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:16 2015 -0500

--

--

[11/22] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[jbellis]

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ec02049
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ec02049
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ec02049

Branch: refs/heads/cassandra-3.0
Commit: 5ec020490f46679a3f13cb96b2f7dfb56744dea2
Parents: 27754c0 5bc56c3
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:16 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:16 2015 -0500

--

--

[15/22] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jbellis]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3f505191
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3f505191
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3f505191

Branch: refs/heads/cassandra-3.0
Commit: 3f50519141df4d0b4c5224e2652c96cfa0333feb
Parents: 1f13528 5ec0204
Author: Jonathan Ellis 
Authored: Fri Sep 4 09:17:26 2015 -0500
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:17:26 2015 -0500

--

--

[07/22] cassandra git commit: refuse to decomission if not in state NORMAL patch by Jan Karlsson and Stefania for CASSANDRA-8741

[jbellis]

refuse to decomission if not in state NORMAL
patch by Jan Karlsson and Stefania for CASSANDRA-8741


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27754c0a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27754c0a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27754c0a

Branch: refs/heads/cassandra-3.0
Commit: 27754c0a41087a2419c09fd99707fd7235cce4ad
Parents: fe80052
Author: Stefania Alborghetti 
Authored: Thu Aug 27 10:01:07 2015 +0800
Committer: Jonathan Ellis 
Committed: Fri Sep 4 09:15:56 2015 -0500

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/service/StorageService.java | 2 ++
 src/java/org/apache/cassandra/tools/NodeTool.java | 3 +++
 3 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 681a40b..6d43c98 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,8 @@
  * Only check KeyCache when it is enabled
  * Change streaming_socket_timeout_in_ms default to 1 hour (CASSANDRA-8611)
  * (cqlsh) update list of CQL keywords (CASSANDRA-9232)
+Merged from 2.0:
+ * Disallow decommission when node is in drained state (CASSANDRA-8741)
 
 
 2.1.9

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 56ddd63..acc82e6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3281,6 +3281,8 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 throw new UnsupportedOperationException("local node is not a 
member of the token ring yet");
 if (tokenMetadata.cloneAfterAllLeft().sortedTokens().size() < 2)
 throw new UnsupportedOperationException("no other normal nodes in 
the ring; decommission would be pointless");
+if (operationMode != Mode.NORMAL)
+throw new UnsupportedOperationException("Node in " + operationMode 
+ " state; wait for status to become normal or restart");
 
 PendingRangeCalculatorService.instance.blockUntilFinished();
 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/27754c0a/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 8e88f45..277f42c 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1471,6 +1471,9 @@ public class NodeTool
 } catch (InterruptedException e)
 {
 throw new RuntimeException("Error decommissioning node", e);
+} catch (UnsupportedOperationException e)
+{
+throw new IllegalStateException("Unsupported operation: " + 
e.getMessage(), e);
 }
 }
 }

[jira] [Commented] (CASSANDRA-10269) Run new upgrade tests on supported upgrade paths

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731027#comment-14731027
 ] 

Jim Witschey commented on CASSANDRA-10269:
--

.bq these are primarily pre to post 8099 tests, so I dont see the point of 3.0 
being the old version.

That's a good point -- [~thobbs] [~iamaleksey] thoughts? If these are general 
upgrade-path tests, then 3.0 -> trunk is important, but if these are specific 
to the new storage engine, I agree with Philip.

.bq I'm fine with only running one on user branches, but if we move these to a 
separate job, then they wont be running on user branches, since those only run 
the normal dtest job.

Running only one of the upgrade paths on dev jobs sounds reasonable, as long as 
trunk is still pre-3.0 -- perhaps we run 2.1 -> HEAD on those branches. I agree 
that the impact on job time will be small.

> Run new upgrade tests on supported upgrade paths
> 
>
> Key: CASSANDRA-10269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10269
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tests
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> The upgrade dtests for 8099 backwards compatibility (originally dealt with in 
> [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/471] and [this 
> JIRA ticket|https://issues.apache.org/jira/browse/CASSANDRA-9893]) need to be 
> run with upgrades over the following upgrade paths:
> - 2.1 -> 3.0
> - 2.2 -> 3.0
> - 3.0 -> trunk
> There are a number of ways we could manage this. We could run the tests as 
> part of the normal dtest jobs in 2.1, 2.2, and 3.0, and select what version 
> to upgrade to based on the version of the test. We could also refactor the 
> new upgrade tests to use the upgrade machinery in the existing upgrade tests.
> [~philipthompson] [~rhatch] Do you have opinions? Can you think of other 
> options?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9921) Combine MV schema definition with MV table definition

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731034#comment-14731034
 ] 

Adam Holmberg commented on CASSANDRA-9921:
--

(ACK) Will likely be starting this Tuesday (8/Sept/15) based on current 
priorities and the US holiday next Monday.

> Combine MV schema definition with MV table definition
> -
>
> Key: CASSANDRA-9921
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9921
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
>  Labels: client-impacting, materializedviews
> Fix For: 3.0.0 rc1
>
>
> Prevent MV from reusing {{system_schema.tables}} and instead move those 
> properties into the {{system_schema.materializedviews}} table to keep them 
> separate entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9921) Combine MV schema definition with MV table definition

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731034#comment-14731034
 ] 

Adam Holmberg edited comment on CASSANDRA-9921 at 9/4/15 4:34 PM:
--

Will likely be starting this Tuesday (8/Sept/15) based on current priorities 
and the US holiday next Monday.


was (Author: aholmber):
(ACK) Will likely be starting this Tuesday (8/Sept/15) based on current 
priorities and the US holiday next Monday.

> Combine MV schema definition with MV table definition
> -
>
> Key: CASSANDRA-9921
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9921
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
>  Labels: client-impacting, materializedviews
> Fix For: 3.0.0 rc1
>
>
> Prevent MV from reusing {{system_schema.tables}} and instead move those 
> properties into the {{system_schema.materializedviews}} table to keep them 
> separate entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10269) Run new upgrade tests on supported upgrade paths

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731027#comment-14731027
 ] 

Jim Witschey edited comment on CASSANDRA-10269 at 9/4/15 4:34 PM:
--

bq. these are primarily pre to post 8099 tests, so I dont see the point of 3.0 
being the old version.

That's a good point -- [~thobbs] [~iamaleksey] thoughts? If these are general 
upgrade-path tests, then 3.0 -> trunk is important, but if these are specific 
to the new storage engine, I agree with Philip.

bq. I'm fine with only running one on user branches, but if we move these to a 
separate job, then they wont be running on user branches, since those only run 
the normal dtest job.

Running only one of the upgrade paths on dev jobs sounds reasonable, as long as 
trunk is still pre-3.0 -- perhaps we run 2.1 -> HEAD on those branches. I agree 
that the impact on job time will be small.


was (Author: mambocab):
.bq these are primarily pre to post 8099 tests, so I dont see the point of 3.0 
being the old version.

That's a good point -- [~thobbs] [~iamaleksey] thoughts? If these are general 
upgrade-path tests, then 3.0 -> trunk is important, but if these are specific 
to the new storage engine, I agree with Philip.

.bq I'm fine with only running one on user branches, but if we move these to a 
separate job, then they wont be running on user branches, since those only run 
the normal dtest job.

Running only one of the upgrade paths on dev jobs sounds reasonable, as long as 
trunk is still pre-3.0 -- perhaps we run 2.1 -> HEAD on those branches. I agree 
that the impact on job time will be small.

> Run new upgrade tests on supported upgrade paths
> 
>
> Key: CASSANDRA-10269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10269
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tests
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> The upgrade dtests for 8099 backwards compatibility (originally dealt with in 
> [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/471] and [this 
> JIRA ticket|https://issues.apache.org/jira/browse/CASSANDRA-9893]) need to be 
> run with upgrades over the following upgrade paths:
> - 2.1 -> 3.0
> - 2.2 -> 3.0
> - 3.0 -> trunk
> There are a number of ways we could manage this. We could run the tests as 
> part of the normal dtest jobs in 2.1, 2.2, and 3.0, and select what version 
> to upgrade to based on the version of the test. We could also refactor the 
> new upgrade tests to use the upgrade machinery in the existing upgrade tests.
> [~philipthompson] [~rhatch] Do you have opinions? Can you think of other 
> options?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9921) Combine MV schema definition with MV table definition

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731035#comment-14731035
 ] 

Adam Holmberg commented on CASSANDRA-9921:
--

Olivier is out through next week so [~adutra] might be interested.

> Combine MV schema definition with MV table definition
> -
>
> Key: CASSANDRA-9921
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9921
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
>  Labels: client-impacting, materializedviews
> Fix For: 3.0.0 rc1
>
>
> Prevent MV from reusing {{system_schema.tables}} and instead move those 
> properties into the {{system_schema.materializedviews}} table to keep them 
> separate entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10269) Run new upgrade tests on supported upgrade paths

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731042#comment-14731042
 ] 

Philip Thompson commented on CASSANDRA-10269:
-

Josh raises a good point that we don't bucketize on windows, in which case, for 
that OS we should split them out into a separate job probably.

> Run new upgrade tests on supported upgrade paths
> 
>
> Key: CASSANDRA-10269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10269
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tests
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> The upgrade dtests for 8099 backwards compatibility (originally dealt with in 
> [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/471] and [this 
> JIRA ticket|https://issues.apache.org/jira/browse/CASSANDRA-9893]) need to be 
> run with upgrades over the following upgrade paths:
> - 2.1 -> 3.0
> - 2.2 -> 3.0
> - 3.0 -> trunk
> There are a number of ways we could manage this. We could run the tests as 
> part of the normal dtest jobs in 2.1, 2.2, and 3.0, and select what version 
> to upgrade to based on the version of the test. We could also refactor the 
> new upgrade tests to use the upgrade machinery in the existing upgrade tests.
> [~philipthompson] [~rhatch] Do you have opinions? Can you think of other 
> options?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10265) Properly deserialize PREPARE_GLOBAL_MESSAGE and update NEWS.txt about repair defaults in 2.2

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730924#comment-14730924
 ] 

Marcus Eriksson commented on CASSANDRA-10265:
-

I have no idea how 
[this|https://github.com/riptano/cassandra-dtest/blob/master/incremental_repair_test.py#L26]
 is not catching it, but I'll have a look

> Properly deserialize PREPARE_GLOBAL_MESSAGE and update NEWS.txt about repair 
> defaults in 2.2
> 
>
> Key: CASSANDRA-10265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10265
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.2.x
>
>
> We deserialize all PREPARE_MESSAGE as non-global, meaning we don't do any 
> anticompaction after incremental repairs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10267) Failing tests in upgrade_trests.paging_test

[Sylvain Lebresne (JIRA)]

Sylvain Lebresne created CASSANDRA-10267:


 Summary: Failing tests in upgrade_trests.paging_test
 Key: CASSANDRA-10267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10267
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Sylvain Lebresne
Assignee: Blake Eggleston
 Fix For: 3.0.0 rc1


This is a continuation of CASSANDRA-9893 to deal with the failure of the 
{{upgrade_trests.paging_test}} tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10269) Run new upgrade tests on supported upgrade paths

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730982#comment-14730982
 ] 

Philip Thompson commented on CASSANDRA-10269:
-

Now that we bucketize, we could easily run them as part of the normal jobs. 
That would probably be best. The question is, what do we set *old* version to 
when running on 3.0?

> Run new upgrade tests on supported upgrade paths
> 
>
> Key: CASSANDRA-10269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10269
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tests
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> The upgrade dtests for 8099 backwards compatibility (originally dealt with in 
> [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/471] and [this 
> JIRA ticket|https://issues.apache.org/jira/browse/CASSANDRA-9893]) need to be 
> run with upgrades over the following upgrade paths:
> - 2.1 -> 3.0
> - 2.2 -> 3.0
> - 3.0 -> trunk
> There are a number of ways we could manage this. We could run the tests as 
> part of the normal dtest jobs in 2.1, 2.2, and 3.0, and select what version 
> to upgrade to based on the version of the test. We could also refactor the 
> new upgrade tests to use the upgrade machinery in the existing upgrade tests.
> [~philipthompson] [~rhatch] Do you have opinions? Can you think of other 
> options?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10166) Failing tests on cassandra 3.0 branch

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730935#comment-14730935
 ] 

Aleksey Yeschenko commented on CASSANDRA-10166:
---

Moving to rc1, as the only remaining subtask (CASSANDRA-10267) is rc1-labeled.

> Failing tests on cassandra 3.0 branch
> -
>
> Key: CASSANDRA-10166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10166
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> Until we find a better way to track those things, this is meant as a master 
> ticket to track tickets open regarding tests (unit test and dtests, though at 
> the time of this writing only dtest are still failing) that are still 
> failing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix test build

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 9658ee9fa -> 8f249a67d


Fix test build


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8f249a67
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8f249a67
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8f249a67

Branch: refs/heads/cassandra-3.0
Commit: 8f249a67d99b11367ff28be86443e20d61caba47
Parents: 9658ee9
Author: Sylvain Lebresne 
Authored: Fri Sep 4 17:30:41 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Sep 4 17:30:41 2015 +0200

--
 .../apache/cassandra/db/SinglePartitionSliceCommandTest.java | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8f249a67/test/unit/org/apache/cassandra/db/SinglePartitionSliceCommandTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/SinglePartitionSliceCommandTest.java 
b/test/unit/org/apache/cassandra/db/SinglePartitionSliceCommandTest.java
index 15b566e..4b7f15a 100644
--- a/test/unit/org/apache/cassandra/db/SinglePartitionSliceCommandTest.java
+++ b/test/unit/org/apache/cassandra/db/SinglePartitionSliceCommandTest.java
@@ -30,7 +30,7 @@ import org.apache.cassandra.db.rows.Cell;
 import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.db.rows.UnfilteredRowIterator;
 import org.apache.cassandra.exceptions.ConfigurationException;
-import org.apache.cassandra.io.util.ByteBufferDataInput;
+import org.apache.cassandra.io.util.DataInputBuffer;
 import org.apache.cassandra.io.util.DataInputPlus;
 import org.apache.cassandra.io.util.DataOutputBuffer;
 import org.apache.cassandra.net.MessagingService;
@@ -99,7 +99,7 @@ public class SinglePartitionSliceCommandTest
 
 DataOutputBuffer out = new DataOutputBuffer((int) 
ReadCommand.legacyReadCommandSerializer.serializedSize(cmd, 
MessagingService.VERSION_21));
 ReadCommand.legacyReadCommandSerializer.serialize(cmd, out, 
MessagingService.VERSION_21);
-DataInputPlus in = new ByteBufferDataInput(out.buffer(), null, 0, 0);
+DataInputPlus in = new DataInputBuffer(out.buffer(), true);
 cmd = ReadCommand.legacyReadCommandSerializer.deserialize(in, 
MessagingService.VERSION_21);
 
 logger.debug("ReadCommand: {}", cmd);
@@ -163,7 +163,7 @@ public class SinglePartitionSliceCommandTest
 response = ReadResponse.createDataResponse(pi, cmd.columnFilter());
 out = new DataOutputBuffer((int) 
ReadResponse.serializer.serializedSize(response, MessagingService.VERSION_30));
 ReadResponse.serializer.serialize(response, out, 
MessagingService.VERSION_30);
-in = new ByteBufferDataInput(out.buffer(), null, 0, 0);
+in = new DataInputBuffer(out.buffer(), true);
 dst = ReadResponse.serializer.deserialize(in, 
MessagingService.VERSION_30);
 pi = dst.makeIterator(cfm, cmd);
 checkForS(pi);
@@ -174,7 +174,7 @@ public class SinglePartitionSliceCommandTest
 response = ReadResponse.createDataResponse(pi, cmd.columnFilter());
 out = new DataOutputBuffer((int) 
ReadResponse.serializer.serializedSize(response, MessagingService.VERSION_30));
 ReadResponse.serializer.serialize(response, out, 
MessagingService.VERSION_30);
-in = new ByteBufferDataInput(out.buffer(), null, 0, 0);
+in = new DataInputBuffer(out.buffer(), true);
 dst = ReadResponse.serializer.deserialize(in, 
MessagingService.VERSION_30);
 pi = dst.makeIterator(cfm, cmd);
 checkForS(pi);

[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731001#comment-14731001
 ] 

Tyler Hobbs commented on CASSANDRA-9664:


I pushed another two small commits to the branch: one that throws an IRE if 
query parameters are in the MV SELECT statement, and one that tests that {{IS 
NOT NULL}} is rejected in normal SELECT statements.

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10265) Properly deserialize PREPARE_GLOBAL_MESSAGE and update NEWS.txt about repair defaults in 2.2

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-10265:
---
Since Version: 2.2.1

So after CASSANDRA-9142, incremental repair is not working as expected due to 
not running anti-compaction?
If we couldn't catch that in dtests, I think we need to fix incremental_repair 
dtest as well.

> Properly deserialize PREPARE_GLOBAL_MESSAGE and update NEWS.txt about repair 
> defaults in 2.2
> 
>
> Key: CASSANDRA-10265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10265
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.2.x
>
>
> We deserialize all PREPARE_MESSAGE as non-global, meaning we don't do any 
> anticompaction after incremental repairs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10265) Properly deserialize PREPARE_GLOBAL_MESSAGE and update NEWS.txt about repair defaults in 2.2

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730961#comment-14730961
 ] 

Marcus Eriksson commented on CASSANDRA-10265:
-

also filed CASSANDRA-10268

> Properly deserialize PREPARE_GLOBAL_MESSAGE and update NEWS.txt about repair 
> defaults in 2.2
> 
>
> Key: CASSANDRA-10265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10265
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.2.x
>
>
> We deserialize all PREPARE_MESSAGE as non-global, meaning we don't do any 
> anticompaction after incremental repairs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10269) Run new upgrade tests on supported upgrade paths

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730996#comment-14730996
 ] 

Philip Thompson commented on CASSANDRA-10269:
-

Well these are primarily pre to post 8099 tests, so I dont see the point of 3.0 
being the old version. Also, Sylvain, since we're bucketizing, they shouldn't 
add a noticeable amount of runtime to the dtests. I'm fine with only running 
one on user branches, but if we move these to a separate job, then they wont be 
running on user branches, since those only run the normal dtest job. Thoughts 
Jim?

> Run new upgrade tests on supported upgrade paths
> 
>
> Key: CASSANDRA-10269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10269
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tests
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> The upgrade dtests for 8099 backwards compatibility (originally dealt with in 
> [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/471] and [this 
> JIRA ticket|https://issues.apache.org/jira/browse/CASSANDRA-9893]) need to be 
> run with upgrades over the following upgrade paths:
> - 2.1 -> 3.0
> - 2.2 -> 3.0
> - 3.0 -> trunk
> There are a number of ways we could manage this. We could run the tests as 
> part of the normal dtest jobs in 2.1, 2.2, and 3.0, and select what version 
> to upgrade to based on the version of the test. We could also refactor the 
> new upgrade tests to use the upgrade machinery in the existing upgrade tests.
> [~philipthompson] [~rhatch] Do you have opinions? Can you think of other 
> options?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)