[jira] [Commented] (CASSANDRA-4386) Allow cql to use the IN syntax on secondary index values

[Steven Warren (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877264#comment-14877264
 ] 

Steven Warren commented on CASSANDRA-4386:
--

I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up a primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column row I want to retrieve. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fix number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.

> Allow cql to use the IN syntax on secondary index values
> 
>
> Key: CASSANDRA-4386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4386
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jeremy Hanna
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: cql
>
> Currently CQL has a syntax for using IN to get a set of rows with a set of 
> keys.  This would also be very helpful for use with columns with secondary 
> indexes on them.  Such as:
> {code}
> select * from users where first_name in ('françois','frank');
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-4386) Allow cql to use the IN syntax on secondary index values

[Steven Warren (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877264#comment-14877264
 ] 

Steven Warren edited comment on CASSANDRA-4386 at 9/19/15 6:43 PM:
---

I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up by primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column row I want to retrieve. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fix number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.


was (Author: swarren):
I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up a primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column row I want to retrieve. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fix number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.

> Allow cql to use the IN syntax on secondary index values
> 
>
> Key: CASSANDRA-4386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4386
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jeremy Hanna
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: cql
>
> Currently CQL has a syntax for using IN to get a set of rows with a set of 
> keys.  This would also be very helpful for use with columns with secondary 
> indexes on them.  Such as:
> {code}
> select * from users where first_name in ('françois','frank');
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Jonathan Shook (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877316#comment-14877316
 ] 

Jonathan Shook commented on CASSANDRA-7486:
---

This seems pretty open-and-shut where I would expect a bit more of a nuanced 
test. We've honestly seen G1 be the operative improvement in some cases in the 
field. I'd much prefer to see "needs more analysis" than to see it resolved as 
fixed. CMS will *not* scale with hardware as we go forward. This is not in 
debate.


> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877420#comment-14877420
 ] 

Pavel Yaskevich commented on CASSANDRA-7486:


It would be interesting to know what happens in the longer running test, more 
than 10 minutes let's say, if there is a way to run it... 

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Benedict
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9578) CPU hogging by cassandra while starting.

[pankaj mishra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877433#comment-14877433
 ] 

pankaj mishra commented on CASSANDRA-9578:
--

Earlier we used san as our cassandra data storage but since we changed our 
storage to ssd problem went away. 
We are still using Cassandra server version 2.0.14



> CPU hogging by cassandra while starting.
> 
>
> Key: CASSANDRA-9578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9578
> Project: Cassandra
>  Issue Type: Bug
>Reporter: pankaj mishra
>
> The problem we experience is that just after starting cassandra, the cpu 
> utilization by java program (cassandra service) increases insanely (100-300%) 
> and cassandra server startup process gets delayed forever.  It does not 
> happen every time we start. Many a times it starts just fine.
> We also observed that while cluster is running fine with every node is in UN 
> mode, after one or two days (time period is not fixed) few nodes goes into DN 
> mode in nodetool status on their own, even though we are not reading any data 
> or even writing any data.
> We just wanted to know that is there any corner case bug in cassandra that 
> might be the reason for this kind of behavior?
> The version of cassandra server along with all environment information is 
> given below.
> Cassandra server version  2.0.14
> JDK version 
> java version "1.8.0_45"
> Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
> unama -a
> Linux  2.6.32-504.16.2.el6.x86_64 #1 SMP Wed Apr 22 06:48:29 UTC 2015 x86_64 
> x86_64 x86_64 GNU/Linux
> blockdev --getra /dev/sdb1 (cassandra data directory)
> 128
>  blockdev --getra /dev/sda2  (cassandra commitlog directory)
> 128
> lbs_release -a
> LSB Version:  
> :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
> Distributor ID:   CentOS
> Description:  CentOS release 6.6 (Final)
> Release:  6.6
> Codename: Final
>   
> We are using following version of cassandra on vm with following version of 
> os.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7486) Migrate to G1GC by default

[Phil Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877434#comment-14877434
 ] 

Phil Yang edited comment on CASSANDRA-7486 at 9/20/15 5:31 AM:
---

It seems that the max heap is still 8G in trunk so maybe it is unfair for G1. 
What is the heap size of these tests?


was (Author: yangzhe1991):
It seems that the max heap is still 8G in trunk so may be it is unfair for G1. 
What is the heap size of these tests? i

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Benedict
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Phil Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877434#comment-14877434
 ] 

Phil Yang commented on CASSANDRA-7486:
--

It seems that the max heap is still 8G in trunk so may be it is unfair for G1. 
What is the heap size of these tests? i

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Benedict
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7486) Migrate to G1GC by default

[Jonathan Shook (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877411#comment-14877411
 ] 

Jonathan Shook edited comment on CASSANDRA-7486 at 9/20/15 5:15 AM:


I do believe that there is a gap between the maximum effective CMS heap sizes 
and the minimum effective G1 sizes. I'd estimate it to be about the 14GB - 24GB 
range. Neither does admirably when taxed for GC throughput in that range. Put 
in another way, I've never and would never advocate that someone use G1 with 
less than 24G of heap. In practice, I use it only on systems with 64GB of 
memory, where it is no big deal to give G1 32GB to work with. We have simply 
seen G1 go slower when it doesn't have adequate scratch space. In essence, it 
really likes to have more memory.

We have also seen anecdotal evidence that G1 seems to settle in, performance 
wise, after a warm-up time. It could be that it needs to collect metrics long 
enough under steady state before it learns how to handle GC and heap allocation 
better. This hasn't been proven out definitively, but is strongly evidenced in 
some longer-run workload studies.

I do agree that when you don't really need more than 12GB of heap, CMS will be 
difficult to beat with the appropriate tunings. I'm not really sure what to do 
about the mid-band where neither CMS nor G1 are very happy. We may have to be 
prescriptive in the sense that if you want to use G1, then you should give it 
enough to work with effectively.

Perhaps we need to make the startup scripts source a different GC config file 
depending on the detected memory in the system. I normally configure G1 as a 
sourced (included) file to the -env.sh script, so this would be fairly 
straightforward.

[~ato...@datastax.com], any comments on this?
 


was (Author: jshook):
I do believe that there is a gap between the maximum effective CMS heap sizes 
and the minimum effective G1 sizes. I'd estimate it to be about the 14GB - 24GB 
range. Neither does admirably when taxed for GC throughput in that range. Put 
in another way, I've never and would never advocate that someone use G1 with 
less than 24G of heap. In practice, I use it only on systems with 64GB of 
memory, where it is no big deal to give G1 32GB to work with. We have simply 
seen G1 go slower when it doesn't have adequate scratch space. In essence, it 
really likes to have more memory.

We have also seen anecdotal evidence that G1 seems to settle in, performance 
wise, after a warm-up time. It could be that it needs to collect metrics long 
enough under steady state before it learns how to handle GC and heap allocation 
better. This hasn't been provided definitively, but is strongly evidenced in 
some longer-run workload studies.

I do agree that when you don't really need more than 12GB of heap, CMS will be 
difficult to beat with the appropriate tunings. I'm not really sure what to do 
about the mid-band where neither CMS nor G1 are very happy. We may have to be 
prescriptive in the sense that if you want to use G1, then you should give it 
enough to work with effectively.

Perhaps we need to make the startup scripts source a different GC config file 
depending on the detected memory in the system. I normally configure G1 as a 
sourced (included) file to the -env.sh script, so this would be fairly 
straightforward.

[~ato...@datastax.com], any comments on this?
 

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Benedict
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Jonathan Shook (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877432#comment-14877432
 ] 

Jonathan Shook commented on CASSANDRA-7486:
---

I'd argue that there already is an increase in pain as you try to use more of 
the metal on a node. We've just become acclimated to it. Instead of scaling the 
compute side over the metal, we do silly things like run multiple instances per 
box. Its not really silly if it gets results, but it is an example of where we 
do something tactically, get so used to it as a necessary complexity, and then 
just keep taking for granted that this is how we do it. I personally don't want 
to keep going down this path. So, I am inclined to carry on with the testing 
and characterization, in time. We should compare notes and methods and see what 
can be done to reduce the overall effort.


> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Benedict
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-4386) Allow cql to use the IN syntax on secondary index values

[Steven Warren (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877264#comment-14877264
 ] 

Steven Warren edited comment on CASSANDRA-4386 at 9/19/15 6:47 PM:
---

I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up by primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column value I want to specify. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fixed number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.


was (Author: swarren):
I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up by primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column value I want to specify. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fix number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.

> Allow cql to use the IN syntax on secondary index values
> 
>
> Key: CASSANDRA-4386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4386
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jeremy Hanna
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: cql
>
> Currently CQL has a syntax for using IN to get a set of rows with a set of 
> keys.  This would also be very helpful for use with columns with secondary 
> indexes on them.  Such as:
> {code}
> select * from users where first_name in ('françois','frank');
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-4386) Allow cql to use the IN syntax on secondary index values

[Steven Warren (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877264#comment-14877264
 ] 

Steven Warren edited comment on CASSANDRA-4386 at 9/19/15 6:44 PM:
---

I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up by primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column value I want to specify. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fix number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.


was (Author: swarren):
I'd like to make a case for raising the priority of this JIRA. I have a use 
case which I don't think should be that unusual and where a secondary index IN 
clause would be fantastic.

I have a set of wide rows ordered by an index (cluster key). I have a secondary 
key on the primary column value.

If I want to look up by primary column value among the set of wide rows I have 
to issue a query to each one:

select value from x where pk = :y and primary = :value

This is fine, the set of wide rows tends to be 1-100 rows so it's 1-100 
parallel queries. However, without the in clause it's 1-100 parallel queries 
per primary column row I want to retrieve. It would be much better to just 
write:

select value from x where pk = :y and primary in :values

That gives me a fix number of queries to execute equal to the number of wide 
rows in the set. This also should perform well since I'm specifying the 
partition key and sending the parallel queries to the partition server for each 
one.

> Allow cql to use the IN syntax on secondary index values
> 
>
> Key: CASSANDRA-4386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4386
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jeremy Hanna
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: cql
>
> Currently CQL has a syntax for using IN to get a set of rows with a set of 
> keys.  This would also be very helpful for use with columns with secondary 
> indexes on them.  Such as:
> {code}
> select * from users where first_name in ('françois','frank');
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10231) Null status entries on nodes that crash during decommission of a different node

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877334#comment-14877334
 ] 

Stefania commented on CASSANDRA-10231:
--

I've attached [a patch|https://github.com/stef1927/cassandra/commits/10231-3.0] 
for 3.0 that fixes the dtest, [~jkni] would you mind trying the patch with your 
Jepsen test? It will also log a message for all GOSSIP entries so the log files 
may get a bit bigger but we will have helpful information should the patch not 
work.

The patch basically fixes this exception, which causes any other GOSSIP 
properties applied by {{onChange}} and following STATUS to be skipped:

{code}
ERROR [GossipStage:2] 2015-09-19 14:14:31,007 CassandraDaemon.java:195 - 
Exception in thread Thread[GossipStage:2,5,main]
java.lang.NullPointerException: null
at 
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) 
~[na:1.8.0_60]
at org.apache.cassandra.hints.HintsCatalog.get(HintsCatalog.java:85) 
~[main/:na]
at 
org.apache.cassandra.hints.HintsService.excise(HintsService.java:263) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.excise(StorageService.java:2166) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.excise(StorageService.java:2178) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:2083)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1672) 
~[main/:na]
at 
org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1220) 
~[main/:na]
at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:1202) 
~[main/:na]
at 
org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1159) 
~[main/:na]
at 
org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
 ~[main/:na]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_60]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
{code}

According to the additional GOSSIP trace message that I've added, host id was 
one such property.

> Null status entries on nodes that crash during decommission of a different 
> node
> ---
>
> Key: CASSANDRA-10231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10231
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Joel Knighton
>Assignee: Stefania
> Fix For: 3.0.x
>
>
> This issue is reproducible through a Jepsen test of materialized views that 
> crashes and decommissions nodes throughout the test.
> In a 5 node cluster, if a node crashes at a certain point (unknown) during 
> the decommission of a different node, it may start with a null entry for the 
> decommissioned node like so:
> DN 10.0.0.5 ? 256 ? null rack1
> This entry does not get updated/cleared by gossip. This entry is removed upon 
> a restart of the affected node.
> This issue is further detailed in ticket 
> [10068|https://issues.apache.org/jira/browse/CASSANDRA-10068].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10377) AssertionError: attempted to delete non-existing file CommitLog

[Vovodroid (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877400#comment-14877400
 ] 

Vovodroid commented on CASSANDRA-10377:
---

Here some data from reproduced event:
Log:
{code}
WARN  20:43:25 deleteWithConfirm [CommitLog-5-1442685442574.log] 
WARN  20:43:25 deleteWithConfirm [CommitLog-5-1442685442575.log] 
WARN  20:43:35 deleteWithConfirm [CommitLog-5-1442685442576.log] 
WARN  20:43:35 deleteWithConfirm [CommitLog-5-1442685442577.log] 
WARN  20:43:35 deleteWithConfirm [CommitLog-5-1442685442578.log] 
WARN  20:43:35 deleteWithConfirm [CommitLog-5-1442685442578.log] 
ERROR 20:43:35 ### not exists CommitLog-5-1442685442578.log
{code}

So we see, that there were two deletion of CommitLog-5-1442685442578, first one 
first successful.
Folder commitlog contains two next logs:
CommitLog-5-1442685442579.log
CommitLog-5-1442685442580.log

Member *CommitLogSegmentManager.activeSegments* contains 
*CommitLog-5-1442685442580.log*.

> AssertionError: attempted to delete non-existing file CommitLog
> ---
>
> Key: CASSANDRA-10377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10377
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: CentOS 7.1/x64
>Reporter: Vovodroid
>Priority: Critical
>
> After several hours of script tests (create and drop users, keyspaces and 
> tables) exception is thrown:
> {code}
> ERROR 02:58:39 Failed managing commit log segments. Commit disk failure 
> policy is stop; terminating thread
> java.lang.AssertionError: attempted to delete non-existing file 
> CommitLog-5-1442599226756.log
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:122) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:149) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:314)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager$2.run(CommitLogSegmentManager.java:374)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager$1.runMayThrow(CommitLogSegmentManager.java:155)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
> I added some logs to *deleteWithConfirm* and it showed that this file really 
> was deleted by previous delete action, i.e. it was second attempt to delete 
> the same log. Commit log with next number exists in the same time, so log was 
> switched.
> I disabled assert and it seems to have no no bad effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Jonathan Shook (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877411#comment-14877411
 ] 

Jonathan Shook commented on CASSANDRA-7486:
---

I do believe that there is a gap between the maximum effective CMS heap sizes 
and the minimum effective G1 sizes. I'd estimate it to be about the 14GB - 24GB 
range. Neither does admirably when taxed for GC throughput in that range. Put 
in another way, I've never and would never advocate that someone use G1 with 
less than 24G of heap. In practice, I use it only on systems with 64GB of 
memory, where it is no big deal to give G1 32GB to work with. We have simply 
seen G1 go slower when it doesn't have adequate scratch space. In essence, it 
really likes to have more memory.

We have also seen anecdotal evidence that G1 seems to settle in, performance 
wise, after a warm-up time. It could be that it needs to collect metrics long 
enough under steady state before it learns how to handle GC and heap allocation 
better. This hasn't been provided definitively, but is strongly evidenced in 
some longer-run workload studies.

I do agree that when you don't really need more than 12GB of heap, CMS will be 
difficult to beat with the appropriate tunings. I'm not really sure what to do 
about the mid-band where neither CMS nor G1 are very happy. We may have to be 
prescriptive in the sense that if you want to use G1, then you should give it 
enough to work with effectively.

Perhaps we need to make the startup scripts source a different GC config file 
depending on the detected memory in the system. I normally configure G1 as a 
sourced (included) file to the -env.sh script, so this would be fairly 
straightforward.

[~ato...@datastax.com], any comments on this?
 

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877404#comment-14877404
 ] 

Benedict commented on CASSANDRA-7486:
-

bq. Is the picture equally bleak at RF=3?

[Regrettably 
so|http://cstar.datastax.com/graph?stats=a1eee43a-5f32-11e5-88b3-42010af0688f=op_rate=3_user=1_aggregates=true=0=112.2=0=163663.5]

bq. Do the "2.2 GC" settings include anything other than the defaults from 
cassandra-env.sh? "ps -efw" output is sufficient.

I haven't double checked, I simply copied [~tjake]'s branch and rebased to 
latest 3.0. It looks like it's just 2.2 defaults.

bq. I'd be happy to take a look at the GC logs if they are available.

The thing is, as I say, the _GC_ burden is pretty consistently lower. However 
the application performance is also worse. Indicating the problem isn't the 
collections, but the VM behavioural changes required to enable G1GC. So 
analyzing GC logs is unlikely to deliver much, and figuring out how to modify 
the application to reduce the burden here is unlikely to be a short task (if 
achievable).

bq. This is not in debate.

I'm afraid nothing is not in debate in this world :)

If you mean to say "CMS will *not* scale with _increasingly gigantic heap 
sizes_" then we would probably be in agreement, however with smallish heaps CMS 
works just fine - better, even. If the mid-to-long term goal of Cassandra is to 
have a constant heap burden, i.e. decouple heap requirements from dataset, then 
it doesn't follow that increasing hardware capabilities requires G1GC. There 
are lots of reasons why this _should_ be our goal, and my understanding is 
there is a general consensus on that, but that's a separate debate. 

Certainly we need to do more research, but I will prognosticate briefly: I 
suspect we will find that with very large heaps (16Gb+) and with lots of 
headroom G1GC begins to outperform CMS, especially wrt the most critical of 
metrics, 99.9%ile. However I suspect we will find CMS continues to dominate in 
domains where it can maintain sufficiently low pause times. 

Since many users target the more modest heap sizes, we may find that it makes 
most sense to provide two default configurations, and have the user opt into 
our "default" G1GC settings if they intend to run with a very large heap. If, 
after extensive research, we find that we can confidently predict configs where 
it makes more sense, we should consider doing this automatically in 
cassandra-env.

My suspicion is we won't manage to do this research in time for GA, but that 
doesn't stop us providing the parallel defaults and documentation to make it 
easy for users to enable it.

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7392) Abort in-progress queries that time out

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877285#comment-14877285
 ] 

Stefania commented on CASSANDRA-7392:
-

bq. This comment seems out of place?

Removed, thank you.

bq. What is this change to Slices, is it fixing an NPE? I am guessing you found 
a few things making heavier use of toCQLString.

It fixes a small bug in creating the CQL string, without this change we end up 
with an unnecessary "AND () = ()"

bq. This is random now so the comment is out of date

Comment updated thanks.

bq. If only a fixed number of timed out queries are reported how about only 
storing references to the first N? Allocating an ArrayList per query doesn't 
make much sense if aggregating doesn't really work yet although it's necessary 
to count identical queries.

bq. [Check if debug is enabled before formatting the string and 
logging?|https://github.com/apache/cassandra/compare/trunk...stef1927:7392-3.0#diff-e06002c30313f8ead63ee472617d1b10R147

Both done, thank you.

bq. This is set to 30

Not sure I understand, perhaps the link is wrong?

bq. I think the utests would be closer to trunk if you rebased. The dtests will 
get a lot worse though.

So far I've kept the 7392-3.0 branch up-to-date with cassandra-3.0 since I 
assumed it would be committed on this branch. I've also resurrected the old 
7392 branch however, which is now identical to 7392-3.0 but rebased on trunk, 
so we have CI for both. There was only one small conflict in 
{{DatabaseDescriptor}}.

CI links:

http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-7392-3.0-dtest/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-7392-3.0-testall/

http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-7392-dtest/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-7392-testall/

> Abort in-progress queries that time out
> ---
>
> Key: CASSANDRA-7392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7392
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 3.x
>
>
> Currently we drop queries that time out before we get to them (because node 
> is overloaded) but not queries that time out while being processed.  
> (Particularly common for index queries on data that shouldn't be indexed.)  
> Adding the latter and logging when we have to interrupt one gets us a poor 
> man's "slow query log" for free.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10326) Performance is worse in 3.0

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877261#comment-14877261
 ] 

Benedict edited comment on CASSANDRA-10326 at 9/19/15 6:57 PM:
---

So, [this 
comparison|http://cstar.datastax.com/graph?stats=518e5484-5ee3-11e5-b421-42010af0688f=99.9th_latency=1_user=1_aggregates=true=0=865.37=0=158.51]
 is especially interesting. With CASSANDRA-10322 now in, reverting GC settings 
to those we have for 2.2 brings things much closer. 

* Latency drops across the board - we're now better than 2.2 for all latency 
metrics
* Throughput is still marginally worse for mixed, and latest queries
* Performance improves 40% for a point query, and 3.0 is now 10% _faster_ than 
2.2 for these, instead of 20% slower

As such, I would recommend that - until we can assess the GC behavioural 
changes further, we revert CASSANDRA-7486

It's also worth noting that this synchronisation of GC config demonstrates a 
clear uptick in garbage in 3.0, that may explain a majority of the remaining 
performance delta. So it may be worth profiling allocations to see if there are 
any areas where quick wins are possible.


was (Author: benedict):
So, [this 
comparison|http://cstar.datastax.com/graph?stats=518e5484-5ee3-11e5-b421-42010af0688f=99.9th_latency=1_user=1_aggregates=true=0=865.37=0=158.51]
 is especially interesting. With CASSANDRA-10322 now in, reverting GC settings 
to those we have for 2.2 brings things much closer. 

* Latency drops across the board - we're now better than 2.2 for all latency 
metrics
* Throughput is still marginally worse for mixed, and latest queries
* Performance improves 40% for a point query, and 3.0 is now 10% _faster_ than 
2.2 for these, instead of 20% slower

As such, I would recommend that - until we can assess the GC behavioural 
changes further, we revert CASSANDRA-7486

> Performance is worse in 3.0
> ---
>
> Key: CASSANDRA-10326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10326
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benedict
>Priority: Critical
> Fix For: 3.0.x
>
>
> Performance is generally turning out to be worse after 8099, despite a number 
> of unrelated performance enhancements being delivered. This isn't entirely 
> unexpected, given a great deal of time was spent optimising the old code, 
> however things appear worse than we had hoped.
> My expectation was that workloads making extensive use of CQL constructs 
> would be faster post-8099, however the latest tests performed with very large 
> CQL rows, including use of collections, still exhibit performance below that 
> of 2.1 and 2.2. 
> Eventually, as the dataset size grows large enough and the locality of access 
> is just right, the reduction in size of our dataset will yield a window 
> during which some users will perform better due simply to improved page cache 
> hit rates. We seem to see this in some of the tests. However we should be at 
> least as fast (and really faster) off the bat.
> The following are some large partition benchmark results, with as many as 40K 
> rows per partition, running LCS. There are a number of parameters we can 
> modify to see how behaviour changes and under what scenarios we might still 
> be faster, but the picture painted isn't brilliant, and is consistent, so we 
> should really try and figure out what's up before GA.
> [trades-with-flags (collections), 
> blade11b|http://cstar.datastax.com/graph?stats=f0a17292-5a13-11e5-847a-42010af0688f=op_rate=1_user=1_aggregates=true=0=4387.02=0=122951.4]
> [trades-with-flags (collections), 
> blade11|http://cstar.datastax.com/graph?stats=e250-5a13-11e5-ae0d-42010af0688f=op_rate=1_user=1_aggregates=true=0=4424.75=0=130158.6]
> [trades (no collections), 
> blade11|http://cstar.datastax.com/graph?stats=9b7da48e-570c-11e5-90fe-42010af0688f=op_rate=1_user=1_aggregates=true=0=2682.46=0=142547.9]
> [~slebresne]: will you have time to look into this before GA?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7486) Migrate to G1GC by default

[Albert P Tobey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Albert P Tobey updated CASSANDRA-7486:
--
Assignee: Benedict  (was: Albert P Tobey)

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Benedict
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Albert P Tobey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877418#comment-14877418
 ] 

Albert P Tobey commented on CASSANDRA-7486:
---

The main point of switching to G1 was to enable most users to get decent - if 
not the best - performance out of the box without having to guess HEAP_NEWSIZE.

Since nobody has the time or inclination to test/discover further, it might as 
well be rolled back. Users won't notice any difference in pain since there was 
never a release with G1.

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877419#comment-14877419
 ] 

Benedict commented on CASSANDRA-7486:
-

Sounds like we're broadly in agreement then.

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7486) Migrate to G1GC by default

[Jonathan Shook (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877316#comment-14877316
 ] 

Jonathan Shook edited comment on CASSANDRA-7486 at 9/19/15 8:44 PM:


This seems pretty open-and-shut where I would expect a bit more of a nuanced 
test. We've honestly seen G1 be the operative improvement in some cases in the 
field. I'd much prefer to see "needs more analysis" than to see it resolved as 
fixed. CMS will *not* scale with hardware as we go forward. This is not in 
debate.

An, nevermind. I see that is what the status is now.



was (Author: jshook):
This seems pretty open-and-shut where I would expect a bit more of a nuanced 
test. We've honestly seen G1 be the operative improvement in some cases in the 
field. I'd much prefer to see "needs more analysis" than to see it resolved as 
fixed. CMS will *not* scale with hardware as we go forward. This is not in 
debate.


> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-7486) Migrate to G1GC by default

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reopened CASSANDRA-7486:
-

We should revert this patch until we can do further analysis. We have 
significantly worse throughput and latency figures in 3.0, and much of this can 
be explained by the new GC settings. This is with more realistic workloads than 
we have previously benchmarked, and also with the current state of 3.0.

[These 
graphs|http://cstar.datastax.com/graph?stats=518e5484-5ee3-11e5-b421-42010af0688f=op_rate=3_user=1_aggregates=true=0=113.3=0=159847.6]
 paint a bleak picture. Throughput takes a 33% hit for point queries 
(Interestingly, there is a reduction in GC _work_ done, presumably indicating 
that card marking / GC store barriers are to blame). Latency is much worse, and 
much less consistent, as is throughput, for all runs.

We may be able to use this information to counteract the problem, but releasing 
3.0 GA with this change as stands seems premature.

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10323) Add more MaterializedView metrics

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-10323:
---
Labels: lhf  (was: )

> Add more MaterializedView metrics
> -
>
> Key: CASSANDRA-10323
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10323
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
>  Labels: lhf
> Fix For: 3.0.x
>
>
> We need to add more metrics to help understand where time is spent in 
> materialized view writes. We currently track the ratio of async base -> view 
> mutations that fail.
> We should also add
>   * The amount of time spent waiting for the partition lock (contention)
>   * The amount of time spent reading data 
> Any others? 
> [~carlyeks] [~jkni] 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix mixed version read request compatibility for compact static tables

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 cffa93c62 -> aba97fc26


Fix mixed version read request compatibility for compact static tables

patch by Blake Eggleston; reviewed by Aleksey Yeschenko for
CASSANDRA-10373


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aba97fc2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aba97fc2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aba97fc2

Branch: refs/heads/cassandra-3.0
Commit: aba97fc265a796ea0ac76c1a4a4953501fe4ab95
Parents: cffa93c
Author: Blake Eggleston 
Authored: Sat Sep 19 10:45:34 2015 -0700
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 19:15:19 2015 +0100

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/db/ReadCommand.java | 6 ++
 2 files changed, 8 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/aba97fc2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 45fa773..851a19b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.0-rc1
+ * Fix mixed version read request compatibility for compact static tables
+   (CASSANDRA-10373)
  * Fix paging of DISTINCT with static and IN (CASSANDRA-10354)
  * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
columns (CASSANDRA-9664)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/aba97fc2/src/java/org/apache/cassandra/db/ReadCommand.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java 
b/src/java/org/apache/cassandra/db/ReadCommand.java
index 1c1da42..91227cf 100644
--- a/src/java/org/apache/cassandra/db/ReadCommand.java
+++ b/src/java/org/apache/cassandra/db/ReadCommand.java
@@ -1297,6 +1297,12 @@ public abstract class ReadCommand implements ReadQuery
 selectionBuilder.add(cellName.column);
 }
 
+// for compact storage tables without clustering keys, the column 
holding the selected value is named
+// 'value' internally we add it to the selection here to prevent 
errors due to unexpected column names
+// when serializing the initial local data response
+if (metadata.isStaticCompactTable() && clusterings.isEmpty())
+selectionBuilder.addAll(metadata.partitionColumns());
+
 in.readBoolean();  // countCql3Rows
 
 // clusterings cannot include STATIC_CLUSTERING, so if the names 
filter is for static columns, clusterings

[1/2] cassandra git commit: Fix mixed version read request compatibility for compact static tables

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4bb5976a7 -> bf5dd032f


Fix mixed version read request compatibility for compact static tables

patch by Blake Eggleston; reviewed by Aleksey Yeschenko for
CASSANDRA-10373


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aba97fc2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aba97fc2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aba97fc2

Branch: refs/heads/trunk
Commit: aba97fc265a796ea0ac76c1a4a4953501fe4ab95
Parents: cffa93c
Author: Blake Eggleston 
Authored: Sat Sep 19 10:45:34 2015 -0700
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 19:15:19 2015 +0100

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/db/ReadCommand.java | 6 ++
 2 files changed, 8 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/aba97fc2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 45fa773..851a19b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.0-rc1
+ * Fix mixed version read request compatibility for compact static tables
+   (CASSANDRA-10373)
  * Fix paging of DISTINCT with static and IN (CASSANDRA-10354)
  * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
columns (CASSANDRA-9664)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/aba97fc2/src/java/org/apache/cassandra/db/ReadCommand.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java 
b/src/java/org/apache/cassandra/db/ReadCommand.java
index 1c1da42..91227cf 100644
--- a/src/java/org/apache/cassandra/db/ReadCommand.java
+++ b/src/java/org/apache/cassandra/db/ReadCommand.java
@@ -1297,6 +1297,12 @@ public abstract class ReadCommand implements ReadQuery
 selectionBuilder.add(cellName.column);
 }
 
+// for compact storage tables without clustering keys, the column 
holding the selected value is named
+// 'value' internally we add it to the selection here to prevent 
errors due to unexpected column names
+// when serializing the initial local data response
+if (metadata.isStaticCompactTable() && clusterings.isEmpty())
+selectionBuilder.addAll(metadata.partitionColumns());
+
 in.readBoolean();  // countCql3Rows
 
 // clusterings cannot include STATIC_CLUSTERING, so if the names 
filter is for static columns, clusterings

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bf5dd032
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bf5dd032
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bf5dd032

Branch: refs/heads/trunk
Commit: bf5dd032fc7428cf3532a2d684ba07bd563435d4
Parents: 4bb5976 aba97fc
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 19:17:53 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 19:17:53 2015 +0100

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/db/ReadCommand.java | 6 ++
 2 files changed, 8 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf5dd032/CHANGES.txt
--
diff --cc CHANGES.txt
index bd51c80,851a19b..119870d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,6 +1,10 @@@
 +3.2
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0.0-rc1
+  * Fix mixed version read request compatibility for compact static tables
+(CASSANDRA-10373)
   * Fix paging of DISTINCT with static and IN (CASSANDRA-10354)
   * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
 columns (CASSANDRA-9664)

[jira] [Commented] (CASSANDRA-10373) Intermitent failures of upgrade_tests.cql_tests:TestCQL.compact_metadata_test

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877244#comment-14877244
 ] 

Aleksey Yeschenko commented on CASSANDRA-10373:
---

LGTM.

Committed as 
[aba97fc265a796ea0ac76c1a4a4953501fe4ab95|https://github.com/apache/cassandra/commit/aba97fc265a796ea0ac76c1a4a4953501fe4ab95]
 to cassandra-3.0 and merged with trunk.

Thank you.

> Intermitent failures of upgrade_tests.cql_tests:TestCQL.compact_metadata_test
> -
>
> Key: CASSANDRA-10373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10373
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Blake Eggleston
> Fix For: 3.0.0 rc1
>
>
> This ticket is to fix the intermittent failures of 
> {{upgrade_tests.cql_tests:TestCQL.compact_metadata_test}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Fix parsing of index targets in thrift metadata conversion

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 aba97fc26 -> d6e14b34b
  refs/heads/trunk bf5dd032f -> 600e1aa17


Fix parsing of index targets in thrift metadata conversion


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d6e14b34
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d6e14b34
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d6e14b34

Branch: refs/heads/cassandra-3.0
Commit: d6e14b34b0dad387140da3fef8522b3cd215e6d6
Parents: aba97fc
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 19:15:39 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 19:19:03 2015 +0100

--
 src/java/org/apache/cassandra/thrift/ThriftConversion.java | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d6e14b34/src/java/org/apache/cassandra/thrift/ThriftConversion.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftConversion.java 
b/src/java/org/apache/cassandra/thrift/ThriftConversion.java
index 005d6c5..3794fe0 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftConversion.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftConversion.java
@@ -42,6 +42,7 @@ import org.apache.cassandra.locator.LocalStrategy;
 import org.apache.cassandra.schema.*;
 import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.Pair;
 import org.apache.cassandra.utils.UUIDGen;
 
 /**
@@ -590,10 +591,8 @@ public class ThriftConversion
 IndexMetadata matchedIndex = null;
 for (IndexMetadata index : cfMetaData.getIndexes())
 {
-String target = index.options.get(IndexTarget.TARGET_OPTION_NAME);
-Matcher m = CassandraIndex.TARGET_REGEX.matcher(target);
-if (target.equals(column.name.toString()) ||
-(m.matches() && m.group(2).equals(column.name.toString(
+Pair target  = 
CassandraIndex.parseTarget(cfMetaData, index);
+if (target.left.equals(column))
 {
 // we already found an index for this column, we've no option 
but to
 // ignore both of them (and any others we've yet to find)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[samt]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/600e1aa1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/600e1aa1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/600e1aa1

Branch: refs/heads/trunk
Commit: 600e1aa1795bb9fecb71e05be42b22988026059f
Parents: bf5dd03 d6e14b3
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 19:21:14 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 19:21:14 2015 +0100

--
 src/java/org/apache/cassandra/thrift/ThriftConversion.java | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)
--

[2/3] cassandra git commit: Fix parsing of index targets in thrift metadata conversion

[samt]

Fix parsing of index targets in thrift metadata conversion


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d6e14b34
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d6e14b34
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d6e14b34

Branch: refs/heads/trunk
Commit: d6e14b34b0dad387140da3fef8522b3cd215e6d6
Parents: aba97fc
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 19:15:39 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 19:19:03 2015 +0100

--
 src/java/org/apache/cassandra/thrift/ThriftConversion.java | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d6e14b34/src/java/org/apache/cassandra/thrift/ThriftConversion.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftConversion.java 
b/src/java/org/apache/cassandra/thrift/ThriftConversion.java
index 005d6c5..3794fe0 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftConversion.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftConversion.java
@@ -42,6 +42,7 @@ import org.apache.cassandra.locator.LocalStrategy;
 import org.apache.cassandra.schema.*;
 import org.apache.cassandra.serializers.MarshalException;
 import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.Pair;
 import org.apache.cassandra.utils.UUIDGen;
 
 /**
@@ -590,10 +591,8 @@ public class ThriftConversion
 IndexMetadata matchedIndex = null;
 for (IndexMetadata index : cfMetaData.getIndexes())
 {
-String target = index.options.get(IndexTarget.TARGET_OPTION_NAME);
-Matcher m = CassandraIndex.TARGET_REGEX.matcher(target);
-if (target.equals(column.name.toString()) ||
-(m.matches() && m.group(2).equals(column.name.toString(
+Pair target  = 
CassandraIndex.parseTarget(cfMetaData, index);
+if (target.left.equals(column))
 {
 // we already found an index for this column, we've no option 
but to
 // ignore both of them (and any others we've yet to find)

[jira] [Commented] (CASSANDRA-10326) Performance is worse in 3.0

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877261#comment-14877261
 ] 

Benedict commented on CASSANDRA-10326:
--

So, [this 
comparison|http://cstar.datastax.com/graph?stats=518e5484-5ee3-11e5-b421-42010af0688f=99.9th_latency=1_user=1_aggregates=true=0=865.37=0=158.51]
 is especially interesting. With CASSANDRA-10322 now in, reverting GC settings 
to those we have for 2.2 brings things much closer. 

* Latency drops across the board - we're now better than 2.2 for all latency 
metrics
* Throughput is still marginally worse for mixed, and latest queries
* Performance improves 40% for a point query, and 3.0 is now 10% _faster_ than 
2.2 for these, instead of 20% slower

As such, I would recommend that - until we can assess the GC behavioural 
changes further, we revert CASSANDRA-7486

> Performance is worse in 3.0
> ---
>
> Key: CASSANDRA-10326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10326
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benedict
>Priority: Critical
> Fix For: 3.0.x
>
>
> Performance is generally turning out to be worse after 8099, despite a number 
> of unrelated performance enhancements being delivered. This isn't entirely 
> unexpected, given a great deal of time was spent optimising the old code, 
> however things appear worse than we had hoped.
> My expectation was that workloads making extensive use of CQL constructs 
> would be faster post-8099, however the latest tests performed with very large 
> CQL rows, including use of collections, still exhibit performance below that 
> of 2.1 and 2.2. 
> Eventually, as the dataset size grows large enough and the locality of access 
> is just right, the reduction in size of our dataset will yield a window 
> during which some users will perform better due simply to improved page cache 
> hit rates. We seem to see this in some of the tests. However we should be at 
> least as fast (and really faster) off the bat.
> The following are some large partition benchmark results, with as many as 40K 
> rows per partition, running LCS. There are a number of parameters we can 
> modify to see how behaviour changes and under what scenarios we might still 
> be faster, but the picture painted isn't brilliant, and is consistent, so we 
> should really try and figure out what's up before GA.
> [trades-with-flags (collections), 
> blade11b|http://cstar.datastax.com/graph?stats=f0a17292-5a13-11e5-847a-42010af0688f=op_rate=1_user=1_aggregates=true=0=4387.02=0=122951.4]
> [trades-with-flags (collections), 
> blade11|http://cstar.datastax.com/graph?stats=e250-5a13-11e5-ae0d-42010af0688f=op_rate=1_user=1_aggregates=true=0=4424.75=0=130158.6]
> [trades (no collections), 
> blade11|http://cstar.datastax.com/graph?stats=9b7da48e-570c-11e5-90fe-42010af0688f=op_rate=1_user=1_aggregates=true=0=2682.46=0=142547.9]
> [~slebresne]: will you have time to look into this before GA?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876976#comment-14876976
 ] 

Sylvain Lebresne commented on CASSANDRA-9664:
-

+1 on the last branch, but could you replace the calls to the 
{{maybeEscapeQuotedName()}} method in {{IndexTarget.java}} by the new 
{{ColumnIdentifier.toCQLString()}}, as I fear we'll forget to do it otherwise.

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[samt]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c21a2b75
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c21a2b75
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c21a2b75

Branch: refs/heads/trunk
Commit: c21a2b758ba36afe8fc591b0b28be300f5868548
Parents: fe15e6c 41731b8
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 11:54:15 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 11:54:15 2015 +0100

--
 .../org/apache/cassandra/schema/CompressionParams.java  | 12 
 1 file changed, 4 insertions(+), 8 deletions(-)
--

[1/3] cassandra git commit: Ninja remove unused imports

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 e25453ba5 -> 41731b84f
  refs/heads/trunk fe15e6c3c -> c21a2b758


Ninja remove unused imports


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41731b84
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41731b84
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41731b84

Branch: refs/heads/cassandra-3.0
Commit: 41731b84fd51bcd6bf1cb17ea22eaa6877df23a0
Parents: e25453b
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 11:53:13 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 11:53:13 2015 +0100

--
 .../org/apache/cassandra/schema/CompressionParams.java  | 12 
 1 file changed, 4 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41731b84/src/java/org/apache/cassandra/schema/CompressionParams.java
--
diff --git a/src/java/org/apache/cassandra/schema/CompressionParams.java 
b/src/java/org/apache/cassandra/schema/CompressionParams.java
index 443b6f1..7f46718 100644
--- a/src/java/org/apache/cassandra/schema/CompressionParams.java
+++ b/src/java/org/apache/cassandra/schema/CompressionParams.java
@@ -20,18 +20,16 @@ package org.apache.cassandra.schema;
 import java.io.IOException;
 import java.lang.reflect.InvocationTargetException;
 import java.lang.reflect.Method;
-import java.util.*;
-import java.util.concurrent.TimeUnit;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
 
 import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.ImmutableSet;
-import com.google.common.collect.Sets;
 import org.apache.commons.lang3.builder.EqualsBuilder;
 import org.apache.commons.lang3.builder.HashCodeBuilder;
-import org.slf4j.LoggerFactory;
 import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
-import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ParameterizedClass;
 import org.apache.cassandra.db.TypeSizes;
 import org.apache.cassandra.exceptions.ConfigurationException;
@@ -39,8 +37,6 @@ import org.apache.cassandra.io.IVersionedSerializer;
 import org.apache.cassandra.io.compress.*;
 import org.apache.cassandra.io.util.DataInputPlus;
 import org.apache.cassandra.io.util.DataOutputPlus;
-import org.apache.cassandra.service.ClientWarn;
-import org.apache.cassandra.utils.NoSpamLogger;
 
 import static java.lang.String.format;
 

[2/3] cassandra git commit: Ninja remove unused imports

[samt]

Ninja remove unused imports


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41731b84
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41731b84
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41731b84

Branch: refs/heads/trunk
Commit: 41731b84fd51bcd6bf1cb17ea22eaa6877df23a0
Parents: e25453b
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 11:53:13 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 11:53:13 2015 +0100

--
 .../org/apache/cassandra/schema/CompressionParams.java  | 12 
 1 file changed, 4 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41731b84/src/java/org/apache/cassandra/schema/CompressionParams.java
--
diff --git a/src/java/org/apache/cassandra/schema/CompressionParams.java 
b/src/java/org/apache/cassandra/schema/CompressionParams.java
index 443b6f1..7f46718 100644
--- a/src/java/org/apache/cassandra/schema/CompressionParams.java
+++ b/src/java/org/apache/cassandra/schema/CompressionParams.java
@@ -20,18 +20,16 @@ package org.apache.cassandra.schema;
 import java.io.IOException;
 import java.lang.reflect.InvocationTargetException;
 import java.lang.reflect.Method;
-import java.util.*;
-import java.util.concurrent.TimeUnit;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
 
 import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.ImmutableSet;
-import com.google.common.collect.Sets;
 import org.apache.commons.lang3.builder.EqualsBuilder;
 import org.apache.commons.lang3.builder.HashCodeBuilder;
-import org.slf4j.LoggerFactory;
 import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
-import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ParameterizedClass;
 import org.apache.cassandra.db.TypeSizes;
 import org.apache.cassandra.exceptions.ConfigurationException;
@@ -39,8 +37,6 @@ import org.apache.cassandra.io.IVersionedSerializer;
 import org.apache.cassandra.io.compress.*;
 import org.apache.cassandra.io.util.DataInputPlus;
 import org.apache.cassandra.io.util.DataOutputPlus;
-import org.apache.cassandra.service.ClientWarn;
-import org.apache.cassandra.utils.NoSpamLogger;
 
 import static java.lang.String.format;
 

[jira] [Updated] (CASSANDRA-10375) nodetool repair fails

[Ruggero Marchei (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruggero Marchei updated CASSANDRA-10375:

Description: 
When I'm running a *nodetool repair* it often gets stalled after few seconds:

{code}[2015-09-19 11:12:13,807] Repair session 
479ca1c0-5ebf-11e5-9619-3f4813058061 for range 
(40511972970986385,59154612555757611] failed with error [repair 
#479ca1c0-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(40511972970986385,59154612555757611]] Validation failed in /10.8.34.113 
(progress: 0%)
[2015-09-19 11:12:13,812] Repair session 479cc8d1-5ebf-11e5-9619-3f4813058061 
for range (6553929828848556033,6576029219234973671] failed with error [repair 
#479cc8d1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(6553929828848556033,6576029219234973671]] Validation failed in /10.8.34.113 
(progress: 0%)
{code}

At the same time I have this exception on another node (I'm not actually 
running multiple repairs):
{code}
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,825 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c2c90-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (-2926621365236563900,-2916361392298929067]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c53a1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (8236929501578674892,8238760988019827700]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479cc8d1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (6553929828848556033,6576029219234973671]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479ca1c0-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (40511972970986385,59154612555757611]], /10.8.34.113 
(see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:68,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 

[jira] [Updated] (CASSANDRA-10375) nodetool repair fails

[Ruggero Marchei (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruggero Marchei updated CASSANDRA-10375:

Description: 
When I'm running a *nodetool repair* it gets stalled after few seconds:

{code}[2015-09-19 11:12:13,807] Repair session 
479ca1c0-5ebf-11e5-9619-3f4813058061 for range 
(40511972970986385,59154612555757611] failed with error [repair 
#479ca1c0-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(40511972970986385,59154612555757611]] Validation failed in /10.8.34.113 
(progress: 0%)
[2015-09-19 11:12:13,812] Repair session 479cc8d1-5ebf-11e5-9619-3f4813058061 
for range (6553929828848556033,6576029219234973671] failed with error [repair 
#479cc8d1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(6553929828848556033,6576029219234973671]] Validation failed in /10.8.34.113 
(progress: 0%)
{code}

At the same time I have this exception on another node (I'm not running 
multiple repairs):
{code}
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,825 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c2c90-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (-2926621365236563900,-2916361392298929067]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c53a1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (8236929501578674892,8238760988019827700]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479cc8d1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (6553929828848556033,6576029219234973671]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479ca1c0-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (40511972970986385,59154612555757611]], /10.8.34.113 
(see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:68,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 

[jira] [Comment Edited] (CASSANDRA-10354) Invalid paging with DISTINCT on static and IN

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876983#comment-14876983
 ] 

Sylvain Lebresne edited comment on CASSANDRA-10354 at 9/19/15 11:59 AM:


Ok, so fixing of 
{{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
requires a bit more attention and I'll try to have a closer look, but I still 
thing the commit linked here should be done and it does fix the test from 
CASSANDRA-10352 which is btw not an upgrade test. So I would suggest maybe 
reviewing that fix on its own right and opening a separate issue for 
{{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
since that's more clearly a backward compatibility issue.


was (Author: slebresne):
Ok, so fixing of 
{{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
requires a bit more attention and I'll try to have a closer look, but I still 
thing the commit linked here should be done and it does fix the test from 
CASSANDRA-10356 which is btw not an upgrade test. So I would suggest maybe 
reviewing that fix on its own right and opening a separate issue for 
{{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
since that's more clearly a backward compatibility issue.

> Invalid paging with DISTINCT on static and IN
> -
>
> Key: CASSANDRA-10354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10354
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> Basically, the test from CASSANDRA-10352 happens to fail on 3.0, but not for 
> the reason in CASSANDRA-10352 but rather because it incorrectly return 
> duplicate results. This is also the reason for the failure of 
> {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
> once that test doesn't run into CASSANDRA-10352 (which it currently does).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10375) nodetool repair fails

[Ruggero Marchei (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877084#comment-14877084
 ] 

Ruggero Marchei commented on CASSANDRA-10375:
-

I think I hit [#10057|CASSANDRA-10057]. I'm closing this one as duplicated

> nodetool repair fails
> -
>
> Key: CASSANDRA-10375
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10375
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Tools
> Environment: multi-dc Cassandra 2.2.1 cluster with 6 nodes, 3 per DC
>Reporter: Ruggero Marchei
> Fix For: 2.2.2
>
>
> When I'm running a *nodetool repair* it gets stalled after few seconds:
> {code}[2015-09-19 11:12:13,807] Repair session 
> 479ca1c0-5ebf-11e5-9619-3f4813058061 for range 
> (40511972970986385,59154612555757611] failed with error [repair 
> #479ca1c0-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (40511972970986385,59154612555757611]] Validation failed in /10.8.34.113 
> (progress: 0%)
> [2015-09-19 11:12:13,812] Repair session 479cc8d1-5ebf-11e5-9619-3f4813058061 
> for range (6553929828848556033,6576029219234973671] failed with error [repair 
> #479cc8d1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (6553929828848556033,6576029219234973671]] Validation failed in /10.8.34.113 
> (progress: 0%)
> {code}
> At the same time I have this exception on another node (I'm not running 
> multiple repairs):
> {code}
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,825 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 Validator.java:246 - 
> Failed creating a merkle tree for [repair 
> #479c2c90-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (-2926621365236563900,-2916361392298929067]], /10.8.34.113 (see log for 
> details)
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 
> CassandraDaemon.java:183 - Exception in thread 
> Thread[ValidationExecutor:66,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the 
> same sstables
> at 
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 Validator.java:246 - 
> Failed creating a merkle tree for [repair 
> #479c53a1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (8236929501578674892,8238760988019827700]], /10.8.34.113 (see log for details)
> ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,900 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,900 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,901 
> CassandraDaemon.java:183 - Exception in thread 
> Thread[ValidationExecutor:66,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the 
> same sstables
> at 
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 Validator.java:246 - 
> Failed creating a merkle tree for 

[jira] [Resolved] (CASSANDRA-10375) nodetool repair fails

[Ruggero Marchei (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruggero Marchei resolved CASSANDRA-10375.
-
   Resolution: Duplicate
Fix Version/s: 2.2.2

> nodetool repair fails
> -
>
> Key: CASSANDRA-10375
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10375
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Tools
> Environment: multi-dc Cassandra 2.2.1 cluster with 6 nodes, 3 per DC
>Reporter: Ruggero Marchei
> Fix For: 2.2.2
>
>
> When I'm running a *nodetool repair* it gets stalled after few seconds:
> {code}[2015-09-19 11:12:13,807] Repair session 
> 479ca1c0-5ebf-11e5-9619-3f4813058061 for range 
> (40511972970986385,59154612555757611] failed with error [repair 
> #479ca1c0-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (40511972970986385,59154612555757611]] Validation failed in /10.8.34.113 
> (progress: 0%)
> [2015-09-19 11:12:13,812] Repair session 479cc8d1-5ebf-11e5-9619-3f4813058061 
> for range (6553929828848556033,6576029219234973671] failed with error [repair 
> #479cc8d1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (6553929828848556033,6576029219234973671]] Validation failed in /10.8.34.113 
> (progress: 0%)
> {code}
> At the same time I have this exception on another node (I'm not running 
> multiple repairs):
> {code}
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,825 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 Validator.java:246 - 
> Failed creating a merkle tree for [repair 
> #479c2c90-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (-2926621365236563900,-2916361392298929067]], /10.8.34.113 (see log for 
> details)
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 
> CassandraDaemon.java:183 - Exception in thread 
> Thread[ValidationExecutor:66,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the 
> same sstables
> at 
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 Validator.java:246 - 
> Failed creating a merkle tree for [repair 
> #479c53a1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
> (8236929501578674892,8238760988019827700]], /10.8.34.113 (see log for details)
> ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,900 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,900 
> CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
> same sstables
> ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,901 
> CassandraDaemon.java:183 - Exception in thread 
> Thread[ValidationExecutor:66,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the 
> same sstables
> at 
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
>  ~[apache-cassandra-2.2.1.jar:2.2.1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 Validator.java:246 - 
> Failed creating a merkle tree for [repair 
> #479cc8d1-5ebf-11e5-9619-3f4813058061 on 

[jira] [Updated] (CASSANDRA-10166) Failing tests on cassandra 3.0 branch

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10166:
--
Fix Version/s: (was: 3.0.0 rc1)
   3.0.x

> Failing tests on cassandra 3.0 branch
> -
>
> Key: CASSANDRA-10166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10166
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sylvain Lebresne
> Fix For: 3.0.x
>
>
> Until we find a better way to track those things, this is meant as a master 
> ticket to track tickets open regarding tests (unit test and dtests, though at 
> the time of this writing only dtest are still failing) that are still 
> failing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10376) Fix upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10376:
--
Issue Type: Sub-task  (was: Bug)
Parent: CASSANDRA-10166

> Fix upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test
> --
>
> Key: CASSANDRA-10376
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10376
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Aleksey Yeschenko
>Assignee: Blake Eggleston
> Fix For: 3.0.x
>
>
> Follow-up to CASSANDRA-10354 to fix the related upgrade issue.
> To quote [~bdeggleston]:
> {quote}
> So the failure is caused by an edge case where a names filter is used in a 
> paging query against a table that needs SinglePartitionNamesCommand instances 
> converted to SinglePartitionSliceCommand instances in order to be converted 
> to legacy read commands.
> If the previous read returned all requested clusterings and a number of rows 
> equal to the page size, the subsequent read would have an empty clustering 
> names filter. When an empty clustering names filter is converted to a slice 
> filter, the slice filter is created with Slices.ALL.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10376) Fix upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877088#comment-14877088
 ] 

Aleksey Yeschenko commented on CASSANDRA-10376:
---

To quote [~bdeggleston] again:
{quote}
Here is the commit that fixes the failure. This test is still failing, but this 
time it's because there's a NPE on the 2.1 node, although the test itself isn't 
failing any assertions.

Regarding the NPE, the cause appears to be the 3.0 node sending the normal 
cells for a partition in the first read, then only the static cells for the 
same partition in the subsequent reads, which causes 
AbstractQueryPager.firstNonStaticCell to return null.
{quote}

> Fix upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test
> --
>
> Key: CASSANDRA-10376
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10376
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Aleksey Yeschenko
>Assignee: Blake Eggleston
> Fix For: 3.0.x
>
>
> Follow-up to CASSANDRA-10354 to fix the related upgrade issue.
> To quote [~bdeggleston]:
> {quote}
> So the failure is caused by an edge case where a names filter is used in a 
> paging query against a table that needs SinglePartitionNamesCommand instances 
> converted to SinglePartitionSliceCommand instances in order to be converted 
> to legacy read commands.
> If the previous read returned all requested clusterings and a number of rows 
> equal to the page size, the subsequent read would have an empty clustering 
> names filter. When an empty clustering names filter is converted to a slice 
> filter, the slice filter is created with Slices.ALL.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9839) Move crc_check_chance out of compressions options

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877023#comment-14877023
 ] 

Sam Tunnicliffe edited comment on CASSANDRA-9839 at 9/19/15 10:55 AM:
--

Thanks, updated bundled python driver and committed in 
{{e25453ba55d35cac802719c6fb906460f13a9c36}}. 
There was another small dtest change required (to {{test_describe}} so I've 
opened a new pull request for that 
([link|https://github.com/riptano/cassandra-dtest/pull/567]). 

edit: plus follow up {{41731b84fd51bcd6bf1cb17ea22eaa6877df23a0}} because I 
didn't see [~carlyeks]'s comment before committing.


was (Author: beobal):
Thanks, updated bundled python driver and committed in 
{{e25453ba55d35cac802719c6fb906460f13a9c36}}. 
There was another small dtest change required (to {{test_describe}} so I've 
opened a new pull request for that 
([link|https://github.com/riptano/cassandra-dtest/pull/567]). 

> Move crc_check_chance out of compressions options
> -
>
> Key: CASSANDRA-9839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9839
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: client-impacting, docs-impacting
> Fix For: 3.0.0 rc1
>
>
> Follow up to CASSANDRA-8384. The option doesn't belong to compression params 
> - it doesn't affect compression, itself, and isn't passed to compressors upon 
> initialization.
> While it's true that it is (currently) only being honored when reading 
> compressed sstables, it still doesn't belong to compression params (and is 
> causing CASSANDRA-7978 -like issues).
> [~tjake] suggested we should make it an option of its own, and I think we 
> should.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10375) nodetool repair fails

[Ruggero Marchei (JIRA)]

Ruggero Marchei created CASSANDRA-10375:
---

 Summary: nodetool repair fails
 Key: CASSANDRA-10375
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10375
 Project: Cassandra
  Issue Type: Bug
  Components: Core, Tools
 Environment: multi-dc Cassandra 2.2.1 cluster with 6 nodes, 3 per DC
Reporter: Ruggero Marchei


When I'm running a *nodetool repair* it often gets stalled after few seconds:

{code}[2015-09-19 11:12:13,807] Repair session 
479ca1c0-5ebf-11e5-9619-3f4813058061 for range 
(40511972970986385,59154612555757611] failed with error [repair 
#479ca1c0-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(40511972970986385,59154612555757611]] Validation failed in /10.8.34.113 
(progress: 0%)
[2015-09-19 11:12:13,812] Repair session 479cc8d1-5ebf-11e5-9619-3f4813058061 
for range (6553929828848556033,6576029219234973671] failed with error [repair 
#479cc8d1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(6553929828848556033,6576029219234973671]] Validation failed in /10.8.34.113 
(progress: 0%)
{code}

At the same time I have this exception on another node:
{code}
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,825 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c2c90-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (-2926621365236563900,-2916361392298929067]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c53a1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (8236929501578674892,8238760988019827700]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479cc8d1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (6553929828848556033,6576029219234973671]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479ca1c0-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (40511972970986385,59154612555757611]], /10.8.34.113 
(see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- 

[jira] [Updated] (CASSANDRA-10375) nodetool repair fails

[Ruggero Marchei (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruggero Marchei updated CASSANDRA-10375:

Description: 
When I'm running a *nodetool repair* it gets stalled after few seconds:

{code}[2015-09-19 11:12:13,807] Repair session 
479ca1c0-5ebf-11e5-9619-3f4813058061 for range 
(40511972970986385,59154612555757611] failed with error [repair 
#479ca1c0-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(40511972970986385,59154612555757611]] Validation failed in /10.8.34.113 
(progress: 0%)
[2015-09-19 11:12:13,812] Repair session 479cc8d1-5ebf-11e5-9619-3f4813058061 
for range (6553929828848556033,6576029219234973671] failed with error [repair 
#479cc8d1-5ebf-11e5-9619-3f4813058061 on static_assets/assets, 
(6553929828848556033,6576029219234973671]] Validation failed in /10.8.34.113 
(progress: 0%)
{code}

At the same time I have this exception on another node (I'm not actually 
running multiple repairs):
{code}
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,825 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c2c90-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (-2926621365236563900,-2916361392298929067]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,826 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,900 Validator.java:246 - 
Failed creating a merkle tree for [repair #479c53a1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (8236929501578674892,8238760988019827700]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,900 
CompactionManager.java:1070 - Cannot start multiple repair sessions over the 
same sstables
ERROR [ValidationExecutor:66] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:66,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1071)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:94)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at 
org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:669)
 ~[apache-cassandra-2.2.1.jar:2.2.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479cc8d1-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (6553929828848556033,6576029219234973671]], 
/10.8.34.113 (see log for details)
ERROR [ValidationExecutor:67] 2015-09-19 11:12:13,901 Validator.java:246 - 
Failed creating a merkle tree for [repair #479ca1c0-5ebf-11e5-9619-3f4813058061 
on static_assets/assets, (40511972970986385,59154612555757611]], /10.8.34.113 
(see log for details)
ERROR [ValidationExecutor:68] 2015-09-19 11:12:13,901 CassandraDaemon.java:183 
- Exception in thread Thread[ValidationExecutor:68,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 

[jira] [Commented] (CASSANDRA-10354) Invalid paging with DISTINCT on static and IN

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876983#comment-14876983
 ] 

Sylvain Lebresne commented on CASSANDRA-10354:
--

Ok, so fixing of 
{{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
requires a bit more attention and I'll try to have a closer look, but I still 
thing the commit linked here should be done and it does fix the test from 
CASSANDRA-10356 which is btw not an upgrade test. So I would suggest maybe 
reviewing that fix on its own right and opening a separate issue for 
{{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
since that's more clearly a backward compatibility issue.

> Invalid paging with DISTINCT on static and IN
> -
>
> Key: CASSANDRA-10354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10354
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> Basically, the test from CASSANDRA-10352 happens to fail on 3.0, but not for 
> the reason in CASSANDRA-10352 but rather because it incorrectly return 
> duplicate results. This is also the reason for the failure of 
> {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
> once that test doesn't run into CASSANDRA-10352 (which it currently does).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[samt]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fe15e6c3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fe15e6c3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fe15e6c3

Branch: refs/heads/trunk
Commit: fe15e6c3c4a8bbfedf04b43cc104a1e91ddd8e16
Parents: be8b9f2 e25453b
Author: Sam Tunnicliffe 
Authored: Sat Sep 19 11:47:50 2015 +0100
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 11:47:50 2015 +0100

--
 CHANGES.txt |   1 +
 NEWS.txt|   2 +
 bin/cqlsh.py|   2 +-
 ...ore-3.0.0-alpha3-b6aa814-SNAPSHOT-shaded.jar | Bin 2217119 -> 0 bytes
 ...ore-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar | Bin 0 -> 2282233 bytes
 ...iver-internal-only-3.0.0a2.post0-03085e6.zip | Bin 230830 -> 0 bytes
 ...iver-internal-only-3.0.0a2.post0-379b6f2.zip | Bin 0 -> 230837 bytes
 .../org/apache/cassandra/config/CFMetaData.java |   9 +-
 .../cql3/statements/TableAttributes.java|  26 ++
 .../apache/cassandra/db/ColumnFamilyStore.java  |  61 --
 .../compress/CompressedRandomAccessReader.java  |  10 ++-
 .../cassandra/io/compress/LZ4Compressor.java|   2 +-
 .../io/sstable/format/SSTableReader.java|  17 ++--
 .../cassandra/io/util/IChecksummedFile.java |  27 ++
 .../cassandra/io/util/ICompressedFile.java  |   3 +-
 .../apache/cassandra/io/util/SegmentedFile.java |  15 +++-
 .../cassandra/schema/CompressionParams.java |  78 +
 .../cassandra/schema/LegacySchemaMigrator.java  |   8 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |   4 +
 .../apache/cassandra/schema/TableParams.java|  24 +-
 .../compress/CompressedInputStream.java |   7 +-
 .../compress/CompressedStreamReader.java|   3 +-
 .../apache/cassandra/utils/DefaultInteger.java  |  51 ---
 .../apache/cassandra/utils/DefaultValue.java|  51 +++
 .../miscellaneous/CrcCheckChanceTest.java   |  84 ---
 .../compression/CompressedInputStreamTest.java  |   3 +-
 26 files changed, 308 insertions(+), 180 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fe15e6c3/CHANGES.txt
--
diff --cc CHANGES.txt
index 537eeeb,e55fd0a..6c3ac84
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,5 +1,9 @@@
 +3.2
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0.0-rc1
+  * Move crc_check_chance out of compression options (CASSANDRA-9839)
   * Fix descending iteration past end of BTreeSearchIterator (CASSANDRA-10301)
   * Transfer hints to a different node on decommission (CASSANDRA-10198)
   * Check partition keys for CAS operations during stmt validation 
(CASSANDRA-10338)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fe15e6c3/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--

[1/3] cassandra git commit: Moved crc_check_chance out of compression options

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 aef71696e -> e25453ba5
  refs/heads/trunk be8b9f299 -> fe15e6c3c


Moved crc_check_chance out of compression options

Patch by Paulo Motta; reviewed by Sam Tunnicliffe for CASSANDRA-9839


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e25453ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e25453ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e25453ba

Branch: refs/heads/cassandra-3.0
Commit: e25453ba55d35cac802719c6fb906460f13a9c36
Parents: aef7169
Author: Paulo Motta 
Authored: Wed Sep 2 19:24:47 2015 -0300
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 11:38:55 2015 +0100

--
 CHANGES.txt |   1 +
 NEWS.txt|   2 +
 bin/cqlsh.py|   2 +-
 ...ore-3.0.0-alpha3-b6aa814-SNAPSHOT-shaded.jar | Bin 2217119 -> 0 bytes
 ...ore-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar | Bin 0 -> 2282233 bytes
 ...iver-internal-only-3.0.0a2.post0-03085e6.zip | Bin 230830 -> 0 bytes
 ...iver-internal-only-3.0.0a2.post0-379b6f2.zip | Bin 0 -> 230837 bytes
 .../org/apache/cassandra/config/CFMetaData.java |   9 +-
 .../cql3/statements/TableAttributes.java|  26 ++
 .../apache/cassandra/db/ColumnFamilyStore.java  |  61 --
 .../compress/CompressedRandomAccessReader.java  |  10 ++-
 .../cassandra/io/compress/LZ4Compressor.java|   2 +-
 .../io/sstable/format/SSTableReader.java|  17 ++--
 .../cassandra/io/util/IChecksummedFile.java |  27 ++
 .../cassandra/io/util/ICompressedFile.java  |   3 +-
 .../apache/cassandra/io/util/SegmentedFile.java |  15 +++-
 .../cassandra/schema/CompressionParams.java |  78 +
 .../cassandra/schema/LegacySchemaMigrator.java  |   8 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |   4 +
 .../apache/cassandra/schema/TableParams.java|  24 +-
 .../compress/CompressedInputStream.java |   7 +-
 .../compress/CompressedStreamReader.java|   3 +-
 .../apache/cassandra/utils/DefaultInteger.java  |  51 ---
 .../apache/cassandra/utils/DefaultValue.java|  51 +++
 .../miscellaneous/CrcCheckChanceTest.java   |  84 ---
 .../compression/CompressedInputStreamTest.java  |   3 +-
 26 files changed, 308 insertions(+), 180 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c0cde2..e55fd0a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-rc1
+ * Move crc_check_chance out of compression options (CASSANDRA-9839)
  * Fix descending iteration past end of BTreeSearchIterator (CASSANDRA-10301)
  * Transfer hints to a different node on decommission (CASSANDRA-10198)
  * Check partition keys for CAS operations during stmt validation 
(CASSANDRA-10338)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 94f3c37..924c35f 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -94,6 +94,8 @@ Upgrading
- The `sstable_compression` and `chunk_length_kb` compression options have 
been deprecated.
  The new options are `class` and `chunk_length_in_kb`. Disabling 
compression should now
  be done by setting the new option `enabled` to `false`.
+   - The compression option `crc_check_chance` became a top-level table 
option, but is currently
+ enforced only against tables with enabled compression.
- Only map syntax is now allowed for caching options. 
ALL/NONE/KEYS_ONLY/ROWS_ONLY syntax
  has been deprecated since 2.1.0 and is being removed in 3.0.0.
- The 'index_interval' option for 'CREATE TABLE' statements, which has been 
deprecated

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index 41fa4fd..07968d0 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -1184,7 +1184,7 @@ class Shell(cmd.Cmd):
 return self.get_table_meta(ks, name)
 except ColumnFamilyNotFound:
 try:
-   return self.get_view_meta(ks, name)
+return self.get_view_meta(ks, name)
 except MaterializedViewNotFound:
 raise ObjectNotFound("%r not found in keyspace %r" % (name, 
ks))
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/lib/cassandra-driver-core-3.0.0-alpha3-b6aa814-SNAPSHOT-shaded.jar

[2/3] cassandra git commit: Moved crc_check_chance out of compression options

[samt]

Moved crc_check_chance out of compression options

Patch by Paulo Motta; reviewed by Sam Tunnicliffe for CASSANDRA-9839


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e25453ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e25453ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e25453ba

Branch: refs/heads/trunk
Commit: e25453ba55d35cac802719c6fb906460f13a9c36
Parents: aef7169
Author: Paulo Motta 
Authored: Wed Sep 2 19:24:47 2015 -0300
Committer: Sam Tunnicliffe 
Committed: Sat Sep 19 11:38:55 2015 +0100

--
 CHANGES.txt |   1 +
 NEWS.txt|   2 +
 bin/cqlsh.py|   2 +-
 ...ore-3.0.0-alpha3-b6aa814-SNAPSHOT-shaded.jar | Bin 2217119 -> 0 bytes
 ...ore-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar | Bin 0 -> 2282233 bytes
 ...iver-internal-only-3.0.0a2.post0-03085e6.zip | Bin 230830 -> 0 bytes
 ...iver-internal-only-3.0.0a2.post0-379b6f2.zip | Bin 0 -> 230837 bytes
 .../org/apache/cassandra/config/CFMetaData.java |   9 +-
 .../cql3/statements/TableAttributes.java|  26 ++
 .../apache/cassandra/db/ColumnFamilyStore.java  |  61 --
 .../compress/CompressedRandomAccessReader.java  |  10 ++-
 .../cassandra/io/compress/LZ4Compressor.java|   2 +-
 .../io/sstable/format/SSTableReader.java|  17 ++--
 .../cassandra/io/util/IChecksummedFile.java |  27 ++
 .../cassandra/io/util/ICompressedFile.java  |   3 +-
 .../apache/cassandra/io/util/SegmentedFile.java |  15 +++-
 .../cassandra/schema/CompressionParams.java |  78 +
 .../cassandra/schema/LegacySchemaMigrator.java  |   8 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |   4 +
 .../apache/cassandra/schema/TableParams.java|  24 +-
 .../compress/CompressedInputStream.java |   7 +-
 .../compress/CompressedStreamReader.java|   3 +-
 .../apache/cassandra/utils/DefaultInteger.java  |  51 ---
 .../apache/cassandra/utils/DefaultValue.java|  51 +++
 .../miscellaneous/CrcCheckChanceTest.java   |  84 ---
 .../compression/CompressedInputStreamTest.java  |   3 +-
 26 files changed, 308 insertions(+), 180 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c0cde2..e55fd0a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-rc1
+ * Move crc_check_chance out of compression options (CASSANDRA-9839)
  * Fix descending iteration past end of BTreeSearchIterator (CASSANDRA-10301)
  * Transfer hints to a different node on decommission (CASSANDRA-10198)
  * Check partition keys for CAS operations during stmt validation 
(CASSANDRA-10338)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 94f3c37..924c35f 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -94,6 +94,8 @@ Upgrading
- The `sstable_compression` and `chunk_length_kb` compression options have 
been deprecated.
  The new options are `class` and `chunk_length_in_kb`. Disabling 
compression should now
  be done by setting the new option `enabled` to `false`.
+   - The compression option `crc_check_chance` became a top-level table 
option, but is currently
+ enforced only against tables with enabled compression.
- Only map syntax is now allowed for caching options. 
ALL/NONE/KEYS_ONLY/ROWS_ONLY syntax
  has been deprecated since 2.1.0 and is being removed in 3.0.0.
- The 'index_interval' option for 'CREATE TABLE' statements, which has been 
deprecated

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index 41fa4fd..07968d0 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -1184,7 +1184,7 @@ class Shell(cmd.Cmd):
 return self.get_table_meta(ks, name)
 except ColumnFamilyNotFound:
 try:
-   return self.get_view_meta(ks, name)
+return self.get_view_meta(ks, name)
 except MaterializedViewNotFound:
 raise ObjectNotFound("%r not found in keyspace %r" % (name, 
ks))
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e25453ba/lib/cassandra-driver-core-3.0.0-alpha3-b6aa814-SNAPSHOT-shaded.jar
--
diff --git a/lib/cassandra-driver-core-3.0.0-alpha3-b6aa814-SNAPSHOT-shaded.jar 

[jira] [Comment Edited] (CASSANDRA-9839) Move crc_check_chance out of compressions options

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877023#comment-14877023
 ] 

Sam Tunnicliffe edited comment on CASSANDRA-9839 at 9/19/15 10:50 AM:
--

Thanks, updated bundled python driver and committed in 
{{e25453ba55d35cac802719c6fb906460f13a9c36}}. 
There was another small dtest change required (to {{test_describe}} so I've 
opened a new pull request for that 
([link|https://github.com/riptano/cassandra-dtest/pull/567]). 


was (Author: beobal):
Thanks, updated bundled python driver and committed in {{}}. 
There was another small dtest change required (to {{test_describe}} so I've 
opened a new pull request for that 
([link|https://github.com/riptano/cassandra-dtest/pull/567]). 

> Move crc_check_chance out of compressions options
> -
>
> Key: CASSANDRA-9839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9839
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: client-impacting, docs-impacting
> Fix For: 3.0.0 rc1
>
>
> Follow up to CASSANDRA-8384. The option doesn't belong to compression params 
> - it doesn't affect compression, itself, and isn't passed to compressors upon 
> initialization.
> While it's true that it is (currently) only being honored when reading 
> compressed sstables, it still doesn't belong to compression params (and is 
> causing CASSANDRA-7978 -like issues).
> [~tjake] suggested we should make it an option of its own, and I think we 
> should.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10376) Fix upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test

[Aleksey Yeschenko (JIRA)]

Aleksey Yeschenko created CASSANDRA-10376:
-

 Summary: Fix 
upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test
 Key: CASSANDRA-10376
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10376
 Project: Cassandra
  Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Blake Eggleston
 Fix For: 3.0.x


Follow-up to CASSANDRA-10354 to fix the related upgrade issue.

To quote [~bdeggleston]:
{quote}
So the failure is caused by an edge case where a names filter is used in a 
paging query against a table that needs SinglePartitionNamesCommand instances 
converted to SinglePartitionSliceCommand instances in order to be converted to 
legacy read commands.

If the previous read returned all requested clusterings and a number of rows 
equal to the page size, the subsequent read would have an empty clustering 
names filter. When an empty clustering names filter is converted to a slice 
filter, the slice filter is created with Slices.ALL.
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10354) Invalid paging with DISTINCT on static and IN

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877080#comment-14877080
 ] 

Aleksey Yeschenko commented on CASSANDRA-10354:
---

Branched out CASSANDRA-10376 to deal with the upgrade test.

[~bdeggleston] can you please review Sylvain's original fix here?

> Invalid paging with DISTINCT on static and IN
> -
>
> Key: CASSANDRA-10354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10354
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> Basically, the test from CASSANDRA-10352 happens to fail on 3.0, but not for 
> the reason in CASSANDRA-10352 but rather because it incorrectly return 
> duplicate results. This is also the reason for the failure of 
> {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
> once that test doesn't run into CASSANDRA-10352 (which it currently does).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10355) Flatten DataInputPlus type hierarchy

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877155#comment-14877155
 ] 

Benedict commented on CASSANDRA-10355:
--

I had an (incomplete) hack at this on the plane, and while [the benefit is 
real|http://cstar.datastax.com/graph?stats=f2d98688-5e7a-11e5-bf84-42010af0688f],
 it is not sufficient to make a difference for 3.0.

> Flatten DataInputPlus type hierarchy
> 
>
> Key: CASSANDRA-10355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10355
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.x
>
>
> Almost every instance extends {{RebufferingInputStream}} now, however there 
> are two last implementations hanging around that we can probably do without. 
> If we do so, the majority of method calls to any {{DataInputPlus}} (i.e., the 
> majority of method calls full stop) can be inlined by hotspot. Thus giving us 
> behaviour pretty similar to passing around raw {{ByteBuffer}} objects.
> It's worth a shot, anyway. 
> If it works remotely well, we should consider splitting readUnsignedVInt into 
> its first byte read (which we would aim to inline) and its proceeding larger 
> read (which we would perhaps ask not to be).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/4] cassandra git commit: Allow MV's SELECT to restrict PK columns

[tylerhobbs]

Allow MV's SELECT to restrict PK columns

Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-9664


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a4253b6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a4253b6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a4253b6

Branch: refs/heads/trunk
Commit: 5a4253b6a17de9810fbc4e1c3b8d4980e26adcca
Parents: 41731b8
Author: Tyler Hobbs 
Authored: Sat Sep 19 10:06:45 2015 -0500
Committer: Tyler Hobbs 
Committed: Sat Sep 19 10:06:45 2015 -0500

--
 CHANGES.txt |2 +
 .../apache/cassandra/config/ViewDefinition.java |   74 +-
 .../apache/cassandra/cql3/AbstractMarker.java   |   31 +-
 .../apache/cassandra/cql3/ColumnIdentifier.java |   35 +
 .../org/apache/cassandra/cql3/Constants.java|   26 +-
 src/java/org/apache/cassandra/cql3/Cql.g|   12 +-
 src/java/org/apache/cassandra/cql3/Json.java|   42 +-
 src/java/org/apache/cassandra/cql3/Lists.java   |8 +-
 src/java/org/apache/cassandra/cql3/Maps.java|   18 +-
 .../cassandra/cql3/MultiColumnRelation.java |   28 +-
 .../org/apache/cassandra/cql3/Operator.java |   58 +-
 .../org/apache/cassandra/cql3/Relation.java |   23 +
 src/java/org/apache/cassandra/cql3/Sets.java|8 +-
 .../cassandra/cql3/SingleColumnRelation.java|   30 +
 src/java/org/apache/cassandra/cql3/Term.java|   19 +-
 .../apache/cassandra/cql3/TokenRelation.java|   26 +
 src/java/org/apache/cassandra/cql3/Tuples.java  |   24 +-
 .../org/apache/cassandra/cql3/TypeCast.java |5 +-
 .../org/apache/cassandra/cql3/UserTypes.java|7 +-
 .../cassandra/cql3/functions/FunctionCall.java  |   16 +-
 .../cql3/restrictions/AbstractRestriction.java  |   14 +-
 .../ForwardingPrimaryKeyRestrictions.java   |6 +
 .../restrictions/MultiColumnRestriction.java|   55 +
 .../cql3/restrictions/Restriction.java  |1 +
 .../restrictions/SingleColumnRestriction.java   |   58 +
 .../restrictions/StatementRestrictions.java |   69 +-
 .../cql3/statements/AlterTableStatement.java|2 +-
 .../cql3/statements/CreateViewStatement.java|   74 +-
 .../cassandra/cql3/statements/IndexTarget.java  |   14 +-
 .../cql3/statements/ModificationStatement.java  |2 +-
 .../cql3/statements/ParsedStatement.java|5 +
 .../cql3/statements/SelectStatement.java|   27 +-
 .../cql3/statements/UpdateStatement.java|2 +
 .../cassandra/db/PartitionRangeReadCommand.java |   13 +-
 .../org/apache/cassandra/db/ReadCommand.java|   12 -
 src/java/org/apache/cassandra/db/ReadQuery.java |   24 +-
 .../db/SinglePartitionReadCommand.java  |   25 +-
 .../apache/cassandra/db/filter/RowFilter.java   |   39 +
 .../apache/cassandra/db/view/TemporalRow.java   |   17 +-
 src/java/org/apache/cassandra/db/view/View.java |  156 ++-
 .../apache/cassandra/db/view/ViewBuilder.java   |   15 +-
 .../internal/composites/CompositesSearcher.java |2 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |   16 +-
 .../org/apache/cassandra/cql3/CQLTester.java|   78 ++
 .../cassandra/cql3/ViewFilteringTest.java   | 1292 ++
 .../org/apache/cassandra/cql3/ViewTest.java |4 +-
 .../SelectSingleColumnRelationTest.java |4 +
 47 files changed, 2270 insertions(+), 248 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e55fd0a..e589626 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.0-rc1
+ * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
+   columns (CASSANDRA-9664)
  * Move crc_check_chance out of compression options (CASSANDRA-9839)
  * Fix descending iteration past end of BTreeSearchIterator (CASSANDRA-10301)
  * Transfer hints to a different node on decommission (CASSANDRA-10198)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/config/ViewDefinition.java
--
diff --git a/src/java/org/apache/cassandra/config/ViewDefinition.java 
b/src/java/org/apache/cassandra/config/ViewDefinition.java
index 39695b9..02acc68 100644
--- a/src/java/org/apache/cassandra/config/ViewDefinition.java
+++ b/src/java/org/apache/cassandra/config/ViewDefinition.java
@@ -17,26 +17,35 @@
  */
 package org.apache.cassandra.config;
 
+import java.util.List;
 import java.util.Objects;
 import java.util.UUID;
+import java.util.stream.Collectors;
 
+import org.antlr.runtime.*;
+import org.apache.cassandra.cql3.*;
+import 

[3/3] cassandra git commit: Allow MV's SELECT to restrict PK columns

[tylerhobbs]

Allow MV's SELECT to restrict PK columns

Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-9664


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a4253b6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a4253b6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a4253b6

Branch: refs/heads/cassandra-3.0
Commit: 5a4253b6a17de9810fbc4e1c3b8d4980e26adcca
Parents: 41731b8
Author: Tyler Hobbs 
Authored: Sat Sep 19 10:06:45 2015 -0500
Committer: Tyler Hobbs 
Committed: Sat Sep 19 10:06:45 2015 -0500

--
 CHANGES.txt |2 +
 .../apache/cassandra/config/ViewDefinition.java |   74 +-
 .../apache/cassandra/cql3/AbstractMarker.java   |   31 +-
 .../apache/cassandra/cql3/ColumnIdentifier.java |   35 +
 .../org/apache/cassandra/cql3/Constants.java|   26 +-
 src/java/org/apache/cassandra/cql3/Cql.g|   12 +-
 src/java/org/apache/cassandra/cql3/Json.java|   42 +-
 src/java/org/apache/cassandra/cql3/Lists.java   |8 +-
 src/java/org/apache/cassandra/cql3/Maps.java|   18 +-
 .../cassandra/cql3/MultiColumnRelation.java |   28 +-
 .../org/apache/cassandra/cql3/Operator.java |   58 +-
 .../org/apache/cassandra/cql3/Relation.java |   23 +
 src/java/org/apache/cassandra/cql3/Sets.java|8 +-
 .../cassandra/cql3/SingleColumnRelation.java|   30 +
 src/java/org/apache/cassandra/cql3/Term.java|   19 +-
 .../apache/cassandra/cql3/TokenRelation.java|   26 +
 src/java/org/apache/cassandra/cql3/Tuples.java  |   24 +-
 .../org/apache/cassandra/cql3/TypeCast.java |5 +-
 .../org/apache/cassandra/cql3/UserTypes.java|7 +-
 .../cassandra/cql3/functions/FunctionCall.java  |   16 +-
 .../cql3/restrictions/AbstractRestriction.java  |   14 +-
 .../ForwardingPrimaryKeyRestrictions.java   |6 +
 .../restrictions/MultiColumnRestriction.java|   55 +
 .../cql3/restrictions/Restriction.java  |1 +
 .../restrictions/SingleColumnRestriction.java   |   58 +
 .../restrictions/StatementRestrictions.java |   69 +-
 .../cql3/statements/AlterTableStatement.java|2 +-
 .../cql3/statements/CreateViewStatement.java|   74 +-
 .../cassandra/cql3/statements/IndexTarget.java  |   14 +-
 .../cql3/statements/ModificationStatement.java  |2 +-
 .../cql3/statements/ParsedStatement.java|5 +
 .../cql3/statements/SelectStatement.java|   27 +-
 .../cql3/statements/UpdateStatement.java|2 +
 .../cassandra/db/PartitionRangeReadCommand.java |   13 +-
 .../org/apache/cassandra/db/ReadCommand.java|   12 -
 src/java/org/apache/cassandra/db/ReadQuery.java |   24 +-
 .../db/SinglePartitionReadCommand.java  |   25 +-
 .../apache/cassandra/db/filter/RowFilter.java   |   39 +
 .../apache/cassandra/db/view/TemporalRow.java   |   17 +-
 src/java/org/apache/cassandra/db/view/View.java |  156 ++-
 .../apache/cassandra/db/view/ViewBuilder.java   |   15 +-
 .../internal/composites/CompositesSearcher.java |2 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |   16 +-
 .../org/apache/cassandra/cql3/CQLTester.java|   78 ++
 .../cassandra/cql3/ViewFilteringTest.java   | 1292 ++
 .../org/apache/cassandra/cql3/ViewTest.java |4 +-
 .../SelectSingleColumnRelationTest.java |4 +
 47 files changed, 2270 insertions(+), 248 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e55fd0a..e589626 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.0-rc1
+ * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
+   columns (CASSANDRA-9664)
  * Move crc_check_chance out of compression options (CASSANDRA-9839)
  * Fix descending iteration past end of BTreeSearchIterator (CASSANDRA-10301)
  * Transfer hints to a different node on decommission (CASSANDRA-10198)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/config/ViewDefinition.java
--
diff --git a/src/java/org/apache/cassandra/config/ViewDefinition.java 
b/src/java/org/apache/cassandra/config/ViewDefinition.java
index 39695b9..02acc68 100644
--- a/src/java/org/apache/cassandra/config/ViewDefinition.java
+++ b/src/java/org/apache/cassandra/config/ViewDefinition.java
@@ -17,26 +17,35 @@
  */
 package org.apache.cassandra.config;
 
+import java.util.List;
 import java.util.Objects;
 import java.util.UUID;
+import java.util.stream.Collectors;
 
+import org.antlr.runtime.*;
+import org.apache.cassandra.cql3.*;
+import 

[1/3] cassandra git commit: Allow MV's SELECT to restrict PK columns

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 41731b84f -> 5a4253b6a


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java 
b/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
new file mode 100644
index 000..2d789c3
--- /dev/null
+++ b/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
@@ -0,0 +1,1292 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.*;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import com.datastax.driver.core.exceptions.InvalidQueryException;
+import junit.framework.Assert;
+
+import org.apache.cassandra.db.SystemKeyspace;
+
+public class ViewFilteringTest extends CQLTester
+{
+int protocolVersion = 4;
+private final List views = new ArrayList<>();
+
+@BeforeClass
+public static void startup()
+{
+requireNetwork();
+}
+@Before
+public void begin()
+{
+views.clear();
+}
+
+@After
+public void end() throws Throwable
+{
+for (String viewName : views)
+executeNet(protocolVersion, "DROP MATERIALIZED VIEW " + viewName);
+}
+
+private void createView(String name, String query) throws Throwable
+{
+executeNet(protocolVersion, String.format(query, name));
+// If exception is thrown, the view will not be added to the list; 
since it shouldn't have been created, this is
+// the desired behavior
+views.add(name);
+}
+
+private void dropView(String name) throws Throwable
+{
+executeNet(protocolVersion, "DROP MATERIALIZED VIEW " + name);
+views.remove(name);
+}
+
+@Test
+public void testMVCreationSelectRestrictions() throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int, e int, 
PRIMARY KEY((a, b), c, d))");
+
+execute("USE " + keyspace());
+executeNet(protocolVersion, "USE " + keyspace());
+
+// IS NOT NULL is required on all PK statements that are not otherwise 
restricted
+List badStatements = Arrays.asList(
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE b IS 
NOT NULL AND c IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND c IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND c is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a = ? 
AND b IS NOT NULL AND c is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a = 
blobAsInt(?) AND b IS NOT NULL AND c is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s PRIMARY KEY 
(a, b, c, d)"
+);
+
+for (String badStatement : badStatements)
+{
+try
+{
+createView("mv1_test", badStatement);
+Assert.fail("Create MV statement should have failed due to 
missing IS NOT NULL restriction: " + badStatement);
+}
+catch (InvalidQueryException exc) {}
+}
+
+List goodStatements = Arrays.asList(
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a = 1 
AND b = 1 AND c IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND c = 1 AND d IS NOT NULL PRIMARY KEY ((a, b), c, 
d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND c = 

[2/3] cassandra git commit: Allow MV's SELECT to restrict PK columns

[tylerhobbs]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/db/ReadQuery.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadQuery.java 
b/src/java/org/apache/cassandra/db/ReadQuery.java
index 3abffd5..d1f5272 100644
--- a/src/java/org/apache/cassandra/db/ReadQuery.java
+++ b/src/java/org/apache/cassandra/db/ReadQuery.java
@@ -33,7 +33,7 @@ import org.apache.cassandra.service.pager.PagingState;
  */
 public interface ReadQuery
 {
-public static final ReadQuery EMPTY = new ReadQuery()
+ReadQuery EMPTY = new ReadQuery()
 {
 public ReadOrderGroup startOrderGroup()
 {
@@ -67,6 +67,16 @@ public interface ReadQuery
 {
 return QueryPager.EMPTY;
 }
+
+public boolean selectsKey(DecoratedKey key)
+{
+return false;
+}
+
+public boolean selectsClustering(DecoratedKey key, Clustering 
clustering)
+{
+return false;
+}
 };
 
 /**
@@ -116,4 +126,16 @@ public interface ReadQuery
  * @return The limits for the query.
  */
 public DataLimits limits();
+
+/**
+ * @return true if the read query would select the given key, including 
checks against the row filter, if
+ * checkRowFilter is true
+ */
+public boolean selectsKey(DecoratedKey key);
+
+/**
+ * @return true if the read query would select the given clustering, 
including checks against the row filter, if
+ * checkRowFilter is true
+ */
+public boolean selectsClustering(DecoratedKey key, Clustering clustering);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
--
diff --git a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java 
b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
index 49cf07c..a8e37b4 100644
--- a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
+++ b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
@@ -21,6 +21,7 @@ import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.*;
 
+import com.google.common.collect.Iterables;
 import org.apache.cassandra.cache.IRowCacheEntry;
 import org.apache.cassandra.cache.RowCacheKey;
 import org.apache.cassandra.cache.RowCacheSentinel;
@@ -190,15 +191,23 @@ public abstract class SinglePartitionReadCommand c.selectsKey(key));
+}
+
+public boolean selectsClustering(DecoratedKey key, Clustering 
clustering)
+{
+return Iterables.any(commands, c -> c.selectsClustering(key, 
clustering));
+}
+
 @Override
 public String toString()
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/db/filter/RowFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/RowFilter.java 
b/src/java/org/apache/cassandra/db/filter/RowFilter.java
index b5968d5..0ff30af 100644
--- a/src/java/org/apache/cassandra/db/filter/RowFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/RowFilter.java
@@ -115,6 +115,45 @@ public abstract class RowFilter implements 
Iterable
 public abstract UnfilteredPartitionIterator 
filter(UnfilteredPartitionIterator iter, int nowInSec);
 
 /**
+ * Returns true if all of the expressions within this filter that apply to 
the partition key are satisfied by
+ * the given key, false otherwise.
+ */
+public boolean partitionKeyRestrictionsAreSatisfiedBy(DecoratedKey key, 
AbstractType keyValidator)
+{
+for (Expression e : expressions)
+{
+if (!e.column.isPartitionKey())
+continue;
+
+ByteBuffer value = keyValidator instanceof CompositeType
+ ? ((CompositeType) 
keyValidator).split(key.getKey())[e.column.position()]
+ : key.getKey();
+if (!e.operator().isSatisfiedBy(e.column.type, value, e.value))
+return false;
+}
+return true;
+}
+
+/**
+ * Returns true if all of the expressions within this filter that apply to 
the clustering key are satisfied by
+ * the given Clustering, false otherwise.
+ */
+public boolean clusteringKeyRestrictionsAreSatisfiedBy(Clustering 
clustering)
+{
+for (Expression e : expressions)
+{
+if (!e.column.isClusteringColumn())
+continue;
+
+if (!e.operator().isSatisfiedBy(e.column.type, 
clustering.get(e.column.position()), e.value))
+{
+return false;
+}
+}
+return true;
+}
+
+/**
  * Returns this filter but without the provided expression. This method
  * *assumes* 

[4/4] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[tylerhobbs]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e44e21ce
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e44e21ce
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e44e21ce

Branch: refs/heads/trunk
Commit: e44e21ce5561464f635609d01d76bea7858018f4
Parents: c21a2b7 5a4253b
Author: Tyler Hobbs 
Authored: Sat Sep 19 10:07:31 2015 -0500
Committer: Tyler Hobbs 
Committed: Sat Sep 19 10:07:31 2015 -0500

--
 CHANGES.txt |2 +
 .../apache/cassandra/config/ViewDefinition.java |   74 +-
 .../apache/cassandra/cql3/AbstractMarker.java   |   31 +-
 .../apache/cassandra/cql3/ColumnIdentifier.java |   35 +
 .../org/apache/cassandra/cql3/Constants.java|   26 +-
 src/java/org/apache/cassandra/cql3/Cql.g|   12 +-
 src/java/org/apache/cassandra/cql3/Json.java|   42 +-
 src/java/org/apache/cassandra/cql3/Lists.java   |8 +-
 src/java/org/apache/cassandra/cql3/Maps.java|   18 +-
 .../cassandra/cql3/MultiColumnRelation.java |   28 +-
 .../org/apache/cassandra/cql3/Operator.java |   58 +-
 .../org/apache/cassandra/cql3/Relation.java |   23 +
 src/java/org/apache/cassandra/cql3/Sets.java|8 +-
 .../cassandra/cql3/SingleColumnRelation.java|   30 +
 src/java/org/apache/cassandra/cql3/Term.java|   19 +-
 .../apache/cassandra/cql3/TokenRelation.java|   26 +
 src/java/org/apache/cassandra/cql3/Tuples.java  |   24 +-
 .../org/apache/cassandra/cql3/TypeCast.java |5 +-
 .../org/apache/cassandra/cql3/UserTypes.java|7 +-
 .../cassandra/cql3/functions/FunctionCall.java  |   16 +-
 .../cql3/restrictions/AbstractRestriction.java  |   14 +-
 .../ForwardingPrimaryKeyRestrictions.java   |6 +
 .../restrictions/MultiColumnRestriction.java|   55 +
 .../cql3/restrictions/Restriction.java  |1 +
 .../restrictions/SingleColumnRestriction.java   |   58 +
 .../restrictions/StatementRestrictions.java |   69 +-
 .../cql3/statements/AlterTableStatement.java|2 +-
 .../cql3/statements/CreateViewStatement.java|   74 +-
 .../cassandra/cql3/statements/IndexTarget.java  |   14 +-
 .../cql3/statements/ModificationStatement.java  |2 +-
 .../cql3/statements/ParsedStatement.java|5 +
 .../cql3/statements/SelectStatement.java|   27 +-
 .../cql3/statements/UpdateStatement.java|2 +
 .../cassandra/db/PartitionRangeReadCommand.java |   13 +-
 .../org/apache/cassandra/db/ReadCommand.java|   12 -
 src/java/org/apache/cassandra/db/ReadQuery.java |   24 +-
 .../db/SinglePartitionReadCommand.java  |   25 +-
 .../apache/cassandra/db/filter/RowFilter.java   |   39 +
 .../apache/cassandra/db/view/TemporalRow.java   |   17 +-
 src/java/org/apache/cassandra/db/view/View.java |  156 ++-
 .../apache/cassandra/db/view/ViewBuilder.java   |   15 +-
 .../internal/composites/CompositesSearcher.java |2 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |   16 +-
 .../org/apache/cassandra/cql3/CQLTester.java|   78 ++
 .../cassandra/cql3/ViewFilteringTest.java   | 1292 ++
 .../org/apache/cassandra/cql3/ViewTest.java |4 +-
 .../SelectSingleColumnRelationTest.java |4 +
 47 files changed, 2270 insertions(+), 248 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e44e21ce/CHANGES.txt
--
diff --cc CHANGES.txt
index 6c3ac84,e589626..7e7a6c3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,6 +1,10 @@@
 +3.2
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0.0-rc1
+  * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
+columns (CASSANDRA-9664)
   * Move crc_check_chance out of compression options (CASSANDRA-9839)
   * Fix descending iteration past end of BTreeSearchIterator (CASSANDRA-10301)
   * Transfer hints to a different node on decommission (CASSANDRA-10198)

[jira] [Updated] (CASSANDRA-10373) Intermitent failures of upgrade_tests.cql_tests:TestCQL.compact_metadata_test

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10373:
--
Reviewer: Aleksey Yeschenko

> Intermitent failures of upgrade_tests.cql_tests:TestCQL.compact_metadata_test
> -
>
> Key: CASSANDRA-10373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10373
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Blake Eggleston
> Fix For: 3.0.0 rc1
>
>
> This ticket is to fix the intermittent failures of 
> {{upgrade_tests.cql_tests:TestCQL.compact_metadata_test}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877182#comment-14877182
 ] 

Aleksey Yeschenko commented on CASSANDRA-9664:
--

Updated the bundled python- and java-driver in 
{{6af4657a96409e2b5f40822646daf42fae33c11b}}. Merged driver PRs before then.

We should be all good now.

> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add 
> support for a syntax which includes a {{SELECT}} statement, but only allows 
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity 
> with the normal CQL {{SELECT}} statement, specifically simple functions in 
> the selected columns, as well as specifying a {{WHERE}} clause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10354) Invalid paging with DISTINCT on static and IN

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877146#comment-14877146
 ] 

Blake Eggleston commented on CASSANDRA-10354:
-

Right, forgot to mention, +1 on the original fix

> Invalid paging with DISTINCT on static and IN
> -
>
> Key: CASSANDRA-10354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10354
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> Basically, the test from CASSANDRA-10352 happens to fail on 3.0, but not for 
> the reason in CASSANDRA-10352 but rather because it incorrectly return 
> duplicate results. This is also the reason for the failure of 
> {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
> once that test doesn't run into CASSANDRA-10352 (which it currently does).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10242) Validate rack information on startup

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877145#comment-14877145
 ] 

Brandon Williams commented on CASSANDRA-10242:
--

Some kind of -D override would be nice for the rare situation where you do blow 
away the data in a dc or something and want to change the racks, but don't want 
to have to blow away your entire system table and regenerate tokens, etc.

> Validate rack information on startup
> 
>
> Key: CASSANDRA-10242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10242
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
> Fix For: 2.1.x
>
>
> Moving to a new rack means that different data should be stored on a node.  
> We already persist rack information in a system table; we should fail startup 
> if this doesn't match what the snitch thinks it should be.  (Either the 
> snitch is wrong, and needs to be fixed, or the machine has been moved and 
> needs to be rebootstrapped.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877154#comment-14877154
 ] 

Benedict commented on CASSANDRA-9669:
-

I've addressed your nits, but giving the patch another review, I realise that 
the approach I've taken with the new parent compaction strategy isn't going to 
cut it in 2.1. This isn't the complete set of operations that can mark things 
compacting, and if we e.g. redistribute summaries (or attempt an "all sstable" 
operation) we may screw up the state badly. 

In 2.1, we can instead safely mark compacting before we make the sstable 
available in the live set, however this too has problems: any "all sstable" 
operation will fail if flushes have gotten far behind, or secondary index 
flushes are taking too long. So we will probably have to introduce a new of 
sstables into the DataTracker.View that tracks these _almost complete_ 
sstables, and filters them from any "all sstable" operation. This is a really 
rather ugly edge case to behaviour, and in 3.X I'll see if there's anything we 
can do to make it more apparent to consumers of the API.

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
>Priority: Critical
>  Labels: correctness
> Fix For: 3.x, 2.1.x, 2.2.x, 3.0.x
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix paging of DISTINCT with static and IN

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 5a4253b6a -> 91e250146


Fix paging of DISTINCT with static and IN

patch by Sylvain Lebresne; reviewed by Blake Eggleston for
CASSANDRA-10354


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/91e25014
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/91e25014
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/91e25014

Branch: refs/heads/cassandra-3.0
Commit: 91e250146156eda8ef9f89a25b43184d4e49c1b4
Parents: 5a4253b
Author: Sylvain Lebresne 
Authored: Wed Sep 16 16:41:25 2015 +0200
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 16:11:11 2015 +0100

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/db/filter/DataLimits.java| 3 +++
 .../apache/cassandra/service/pager/MultiPartitionPager.java| 6 +++---
 3 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/91e25014/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e589626..45fa773 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-rc1
+ * Fix paging of DISTINCT with static and IN (CASSANDRA-10354)
  * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
columns (CASSANDRA-9664)
  * Move crc_check_chance out of compression options (CASSANDRA-9839)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/91e25014/src/java/org/apache/cassandra/db/filter/DataLimits.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/DataLimits.java 
b/src/java/org/apache/cassandra/db/filter/DataLimits.java
index 75c8290..d5eefe3 100644
--- a/src/java/org/apache/cassandra/db/filter/DataLimits.java
+++ b/src/java/org/apache/cassandra/db/filter/DataLimits.java
@@ -303,7 +303,10 @@ public abstract class DataLimits
 // rows in the partition. However, if we only have the static 
row, it will be returned as one row
 // so count it.
 if (hasLiveStaticRow && rowInCurrentPartition == 0)
+{
 ++rowCounted;
+++rowInCurrentPartition;
+}
 }
 
 public int counted()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/91e25014/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
--
diff --git 
a/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java 
b/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
index ee2db9f..e826be6 100644
--- a/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
+++ b/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
@@ -169,13 +169,13 @@ public class MultiPartitionPager implements QueryPager
 {
 while (result == null || !result.hasNext())
 {
+if (result != null)
+result.close();
+
 // This sets us on the first non-exhausted pager
 if (isExhausted())
 return endOfData();
 
-if (result != null)
-result.close();
-
 int toQuery = pageSize - counter.counted();
 result = consistency == null
? pagers[current].fetchPageInternal(toQuery, orderGroup)

[1/2] cassandra git commit: Update bundled drivers for CASSANDRA-9664

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1e669e562 -> 92b813165


Update bundled drivers for CASSANDRA-9664


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6af4657a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6af4657a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6af4657a

Branch: refs/heads/trunk
Commit: 6af4657a96409e2b5f40822646daf42fae33c11b
Parents: 91e2501
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 16:30:02 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 16:30:02 2015 +0100

--
 ...core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar | Bin 0 -> 2281177 bytes
 ...core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar | Bin 2282233 -> 0 bytes
 ...river-internal-only-3.0.0a2.post0-379b6f2.zip | Bin 230837 -> 0 bytes
 ...river-internal-only-3.0.0a2.post0-b9dfda5.zip | Bin 0 -> 230861 bytes
 4 files changed, 0 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar
--
diff --git a/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar 
b/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar
new file mode 100644
index 000..daf7837
Binary files /dev/null and 
b/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar
--
diff --git a/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar 
b/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar
deleted file mode 100644
index cf30f25..000
Binary files 
a/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar and 
/dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip 
b/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip
deleted file mode 100644
index 5605ef7..000
Binary files a/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip and 
/dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip 
b/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip
new file mode 100644
index 000..20d18eb
Binary files /dev/null and 
b/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip differ

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/92b81316
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/92b81316
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/92b81316

Branch: refs/heads/trunk
Commit: 92b813165484a67069fbd233c8ed1eac7a8a4fe1
Parents: 1e669e5 6af4657
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 16:30:48 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 16:30:48 2015 +0100

--
 ...core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar | Bin 0 -> 2281177 bytes
 ...core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar | Bin 2282233 -> 0 bytes
 ...river-internal-only-3.0.0a2.post0-379b6f2.zip | Bin 230837 -> 0 bytes
 ...river-internal-only-3.0.0a2.post0-b9dfda5.zip | Bin 0 -> 230861 bytes
 4 files changed, 0 insertions(+), 0 deletions(-)
--

[jira] [Created] (CASSANDRA-10377) AssertionError: attempted to delete non-existing file CommitLog

[Vovodroid (JIRA)]

Vovodroid created CASSANDRA-10377:
-

 Summary: AssertionError: attempted to delete non-existing file 
CommitLog
 Key: CASSANDRA-10377
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10377
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: CentOS 7.1/x64
Reporter: Vovodroid
Priority: Critical


After several hours of script tests (create and drop users, keyspaces and 
tables) exception is thrown:
{code}
ERROR 02:58:39 Failed managing commit log segments. Commit disk failure policy 
is stop; terminating thread
java.lang.AssertionError: attempted to delete non-existing file 
CommitLog-5-1442599226756.log
at 
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:122) 
~[main/:na]
at 
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:149) 
~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:314)
 ~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLogSegmentManager$2.run(CommitLogSegmentManager.java:374)
 ~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLogSegmentManager$1.runMayThrow(CommitLogSegmentManager.java:155)
 ~[main/:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
{code}

I added some logs to *deleteWithConfirm* and it showed that this file really 
was deleted by previous delete action, i.e. it was second attempt to delete the 
same log. Commit log with next number exists in the same time, so log was 
switched.

I disabled assert and it seems to have no no bad effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/4] cassandra git commit: Allow MV's SELECT to restrict PK columns

[tylerhobbs]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/db/ReadQuery.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadQuery.java 
b/src/java/org/apache/cassandra/db/ReadQuery.java
index 3abffd5..d1f5272 100644
--- a/src/java/org/apache/cassandra/db/ReadQuery.java
+++ b/src/java/org/apache/cassandra/db/ReadQuery.java
@@ -33,7 +33,7 @@ import org.apache.cassandra.service.pager.PagingState;
  */
 public interface ReadQuery
 {
-public static final ReadQuery EMPTY = new ReadQuery()
+ReadQuery EMPTY = new ReadQuery()
 {
 public ReadOrderGroup startOrderGroup()
 {
@@ -67,6 +67,16 @@ public interface ReadQuery
 {
 return QueryPager.EMPTY;
 }
+
+public boolean selectsKey(DecoratedKey key)
+{
+return false;
+}
+
+public boolean selectsClustering(DecoratedKey key, Clustering 
clustering)
+{
+return false;
+}
 };
 
 /**
@@ -116,4 +126,16 @@ public interface ReadQuery
  * @return The limits for the query.
  */
 public DataLimits limits();
+
+/**
+ * @return true if the read query would select the given key, including 
checks against the row filter, if
+ * checkRowFilter is true
+ */
+public boolean selectsKey(DecoratedKey key);
+
+/**
+ * @return true if the read query would select the given clustering, 
including checks against the row filter, if
+ * checkRowFilter is true
+ */
+public boolean selectsClustering(DecoratedKey key, Clustering clustering);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
--
diff --git a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java 
b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
index 49cf07c..a8e37b4 100644
--- a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
+++ b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
@@ -21,6 +21,7 @@ import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.*;
 
+import com.google.common.collect.Iterables;
 import org.apache.cassandra.cache.IRowCacheEntry;
 import org.apache.cassandra.cache.RowCacheKey;
 import org.apache.cassandra.cache.RowCacheSentinel;
@@ -190,15 +191,23 @@ public abstract class SinglePartitionReadCommand c.selectsKey(key));
+}
+
+public boolean selectsClustering(DecoratedKey key, Clustering 
clustering)
+{
+return Iterables.any(commands, c -> c.selectsClustering(key, 
clustering));
+}
+
 @Override
 public String toString()
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/src/java/org/apache/cassandra/db/filter/RowFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/RowFilter.java 
b/src/java/org/apache/cassandra/db/filter/RowFilter.java
index b5968d5..0ff30af 100644
--- a/src/java/org/apache/cassandra/db/filter/RowFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/RowFilter.java
@@ -115,6 +115,45 @@ public abstract class RowFilter implements 
Iterable
 public abstract UnfilteredPartitionIterator 
filter(UnfilteredPartitionIterator iter, int nowInSec);
 
 /**
+ * Returns true if all of the expressions within this filter that apply to 
the partition key are satisfied by
+ * the given key, false otherwise.
+ */
+public boolean partitionKeyRestrictionsAreSatisfiedBy(DecoratedKey key, 
AbstractType keyValidator)
+{
+for (Expression e : expressions)
+{
+if (!e.column.isPartitionKey())
+continue;
+
+ByteBuffer value = keyValidator instanceof CompositeType
+ ? ((CompositeType) 
keyValidator).split(key.getKey())[e.column.position()]
+ : key.getKey();
+if (!e.operator().isSatisfiedBy(e.column.type, value, e.value))
+return false;
+}
+return true;
+}
+
+/**
+ * Returns true if all of the expressions within this filter that apply to 
the clustering key are satisfied by
+ * the given Clustering, false otherwise.
+ */
+public boolean clusteringKeyRestrictionsAreSatisfiedBy(Clustering 
clustering)
+{
+for (Expression e : expressions)
+{
+if (!e.column.isClusteringColumn())
+continue;
+
+if (!e.operator().isSatisfiedBy(e.column.type, 
clustering.get(e.column.position()), e.value))
+{
+return false;
+}
+}
+return true;
+}
+
+/**
  * Returns this filter but without the provided expression. This method
  * *assumes* 

[1/4] cassandra git commit: Allow MV's SELECT to restrict PK columns

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/trunk c21a2b758 -> e44e21ce5


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a4253b6/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java 
b/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
new file mode 100644
index 000..2d789c3
--- /dev/null
+++ b/test/unit/org/apache/cassandra/cql3/ViewFilteringTest.java
@@ -0,0 +1,1292 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.*;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import com.datastax.driver.core.exceptions.InvalidQueryException;
+import junit.framework.Assert;
+
+import org.apache.cassandra.db.SystemKeyspace;
+
+public class ViewFilteringTest extends CQLTester
+{
+int protocolVersion = 4;
+private final List views = new ArrayList<>();
+
+@BeforeClass
+public static void startup()
+{
+requireNetwork();
+}
+@Before
+public void begin()
+{
+views.clear();
+}
+
+@After
+public void end() throws Throwable
+{
+for (String viewName : views)
+executeNet(protocolVersion, "DROP MATERIALIZED VIEW " + viewName);
+}
+
+private void createView(String name, String query) throws Throwable
+{
+executeNet(protocolVersion, String.format(query, name));
+// If exception is thrown, the view will not be added to the list; 
since it shouldn't have been created, this is
+// the desired behavior
+views.add(name);
+}
+
+private void dropView(String name) throws Throwable
+{
+executeNet(protocolVersion, "DROP MATERIALIZED VIEW " + name);
+views.remove(name);
+}
+
+@Test
+public void testMVCreationSelectRestrictions() throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, c int, d int, e int, 
PRIMARY KEY((a, b), c, d))");
+
+execute("USE " + keyspace());
+executeNet(protocolVersion, "USE " + keyspace());
+
+// IS NOT NULL is required on all PK statements that are not otherwise 
restricted
+List badStatements = Arrays.asList(
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE b IS 
NOT NULL AND c IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND c IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND c is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a = ? 
AND b IS NOT NULL AND c is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a = 
blobAsInt(?) AND b IS NOT NULL AND c is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s PRIMARY KEY 
(a, b, c, d)"
+);
+
+for (String badStatement : badStatements)
+{
+try
+{
+createView("mv1_test", badStatement);
+Assert.fail("Create MV statement should have failed due to 
missing IS NOT NULL restriction: " + badStatement);
+}
+catch (InvalidQueryException exc) {}
+}
+
+List goodStatements = Arrays.asList(
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a = 1 
AND b = 1 AND c IS NOT NULL AND d is NOT NULL PRIMARY KEY ((a, b), c, d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND c = 1 AND d IS NOT NULL PRIMARY KEY ((a, b), c, 
d)",
+"CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE a IS 
NOT NULL AND b IS NOT NULL AND c = 1 AND d 

[1/2] cassandra git commit: Fix paging of DISTINCT with static and IN

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk e44e21ce5 -> 1e669e562


Fix paging of DISTINCT with static and IN

patch by Sylvain Lebresne; reviewed by Blake Eggleston for
CASSANDRA-10354


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/91e25014
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/91e25014
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/91e25014

Branch: refs/heads/trunk
Commit: 91e250146156eda8ef9f89a25b43184d4e49c1b4
Parents: 5a4253b
Author: Sylvain Lebresne 
Authored: Wed Sep 16 16:41:25 2015 +0200
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 16:11:11 2015 +0100

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/db/filter/DataLimits.java| 3 +++
 .../apache/cassandra/service/pager/MultiPartitionPager.java| 6 +++---
 3 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/91e25014/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e589626..45fa773 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.0-rc1
+ * Fix paging of DISTINCT with static and IN (CASSANDRA-10354)
  * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
columns (CASSANDRA-9664)
  * Move crc_check_chance out of compression options (CASSANDRA-9839)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/91e25014/src/java/org/apache/cassandra/db/filter/DataLimits.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/DataLimits.java 
b/src/java/org/apache/cassandra/db/filter/DataLimits.java
index 75c8290..d5eefe3 100644
--- a/src/java/org/apache/cassandra/db/filter/DataLimits.java
+++ b/src/java/org/apache/cassandra/db/filter/DataLimits.java
@@ -303,7 +303,10 @@ public abstract class DataLimits
 // rows in the partition. However, if we only have the static 
row, it will be returned as one row
 // so count it.
 if (hasLiveStaticRow && rowInCurrentPartition == 0)
+{
 ++rowCounted;
+++rowInCurrentPartition;
+}
 }
 
 public int counted()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/91e25014/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
--
diff --git 
a/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java 
b/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
index ee2db9f..e826be6 100644
--- a/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
+++ b/src/java/org/apache/cassandra/service/pager/MultiPartitionPager.java
@@ -169,13 +169,13 @@ public class MultiPartitionPager implements QueryPager
 {
 while (result == null || !result.hasNext())
 {
+if (result != null)
+result.close();
+
 // This sets us on the first non-exhausted pager
 if (isExhausted())
 return endOfData();
 
-if (result != null)
-result.close();
-
 int toQuery = pageSize - counter.counted();
 result = consistency == null
? pagers[current].fetchPageInternal(toQuery, orderGroup)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e669e56
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e669e56
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e669e56

Branch: refs/heads/trunk
Commit: 1e669e5629950df7156cfd482a2c702d804e5ee9
Parents: e44e21c 91e2501
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 16:12:02 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 16:12:02 2015 +0100

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/db/filter/DataLimits.java| 3 +++
 .../apache/cassandra/service/pager/MultiPartitionPager.java| 6 +++---
 3 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e669e56/CHANGES.txt
--
diff --cc CHANGES.txt
index 7e7a6c3,45fa773..bd51c80
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,5 +1,9 @@@
 +3.2
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0.0-rc1
+  * Fix paging of DISTINCT with static and IN (CASSANDRA-10354)
   * Allow MATERIALIZED VIEW's SELECT statement to restrict primary key
 columns (CASSANDRA-9664)
   * Move crc_check_chance out of compression options (CASSANDRA-9839)

[jira] [Commented] (CASSANDRA-10354) Invalid paging with DISTINCT on static and IN

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877171#comment-14877171
 ] 

Aleksey Yeschenko commented on CASSANDRA-10354:
---

Thanks, committed as 
[91e250146156eda8ef9f89a25b43184d4e49c1b4|https://github.com/apache/cassandra/commit/91e250146156eda8ef9f89a25b43184d4e49c1b4]
 to cassandra-3.0 and merged with trunk.

> Invalid paging with DISTINCT on static and IN
> -
>
> Key: CASSANDRA-10354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10354
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc1
>
>
> Basically, the test from CASSANDRA-10352 happens to fail on 3.0, but not for 
> the reason in CASSANDRA-10352 but rather because it incorrectly return 
> duplicate results. This is also the reason for the failure of 
> {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}} 
> once that test doesn't run into CASSANDRA-10352 (which it currently does).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Update bundled drivers for CASSANDRA-9664

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 91e250146 -> 6af4657a9


Update bundled drivers for CASSANDRA-9664


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6af4657a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6af4657a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6af4657a

Branch: refs/heads/cassandra-3.0
Commit: 6af4657a96409e2b5f40822646daf42fae33c11b
Parents: 91e2501
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 16:30:02 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 16:30:02 2015 +0100

--
 ...core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar | Bin 0 -> 2281177 bytes
 ...core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar | Bin 2282233 -> 0 bytes
 ...river-internal-only-3.0.0a2.post0-379b6f2.zip | Bin 230837 -> 0 bytes
 ...river-internal-only-3.0.0a2.post0-b9dfda5.zip | Bin 0 -> 230861 bytes
 4 files changed, 0 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar
--
diff --git a/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar 
b/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar
new file mode 100644
index 000..daf7837
Binary files /dev/null and 
b/lib/cassandra-driver-core-3.0.0-alpha3-093a692-SNAPSHOT-shaded.jar differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar
--
diff --git a/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar 
b/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar
deleted file mode 100644
index cf30f25..000
Binary files 
a/lib/cassandra-driver-core-3.0.0-alpha3-f9b7e7c-SNAPSHOT-shaded.jar and 
/dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip 
b/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip
deleted file mode 100644
index 5605ef7..000
Binary files a/lib/cassandra-driver-internal-only-3.0.0a2.post0-379b6f2.zip and 
/dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af4657a/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip 
b/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip
new file mode 100644
index 000..20d18eb
Binary files /dev/null and 
b/lib/cassandra-driver-internal-only-3.0.0a2.post0-b9dfda5.zip differ

[jira] [Assigned] (CASSANDRA-10111) reconnecting snitch can bypass cluster name check

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-10111:


Assignee: Stefania

> reconnecting snitch can bypass cluster name check
> -
>
> Key: CASSANDRA-10111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10111
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: 2.0.x
>Reporter: Chris Burroughs
>Assignee: Stefania
>  Labels: gossip
> Fix For: 2.1.x
>
>
> Setup:
>  * Two clusters: A & B
>  * Both are two DC cluster
>  * Both use GossipingPropertyFileSnitch with different 
> listen_address/broadcast_address
> A new node was added to cluster A with a broadcast_address of an existing 
> node in cluster B (due to an out of data DNS entry).  Cluster B  added all of 
> the nodes from cluster A, somehow bypassing the cluster name mismatch check 
> for this nodes.  The first reference to cluster A nodes in cluster B logs is 
> when then were added:
> {noformat}
>  INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node 
> /8.37.70.168 is now part of the cluster
> {noformat}
> Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept 
> them out with 'ClusterName mismatch'.  Cluster B however tried to send to 
> send reads/writes to cluster A and general mayhem ensued.
> Obviously this is a Bad (TM) config that Should Not Be Done.  However, since 
> the consequence of crazy merged clusters are really bad (the reason there is 
> the name mismatch check in the first place) I think the hole is reasonable to 
> plug.  I'm not sure exactly what the code path is that skips the check in 
> GossipDigestSynVerbHandler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10111) reconnecting snitch can bypass cluster name check

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877217#comment-14877217
 ] 

Stefania commented on CASSANDRA-10111:
--

I'll take it but I have a few critical tickets that have higher priority.

> reconnecting snitch can bypass cluster name check
> -
>
> Key: CASSANDRA-10111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10111
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: 2.0.x
>Reporter: Chris Burroughs
>  Labels: gossip
> Fix For: 2.1.x
>
>
> Setup:
>  * Two clusters: A & B
>  * Both are two DC cluster
>  * Both use GossipingPropertyFileSnitch with different 
> listen_address/broadcast_address
> A new node was added to cluster A with a broadcast_address of an existing 
> node in cluster B (due to an out of data DNS entry).  Cluster B  added all of 
> the nodes from cluster A, somehow bypassing the cluster name mismatch check 
> for this nodes.  The first reference to cluster A nodes in cluster B logs is 
> when then were added:
> {noformat}
>  INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node 
> /8.37.70.168 is now part of the cluster
> {noformat}
> Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept 
> them out with 'ClusterName mismatch'.  Cluster B however tried to send to 
> send reads/writes to cluster A and general mayhem ensued.
> Obviously this is a Bad (TM) config that Should Not Be Done.  However, since 
> the consequence of crazy merged clusters are really bad (the reason there is 
> the name mismatch check in the first place) I think the hole is reasonable to 
> plug.  I'm not sure exactly what the code path is that skips the check in 
> GossipDigestSynVerbHandler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10369) cqlsh prompt includes name of keyspace after failed `use` statement

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp reassigned CASSANDRA-10369:


Assignee: Robert Stupp

> cqlsh prompt includes name of keyspace after failed `use` statement
> ---
>
> Key: CASSANDRA-10369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10369
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Robert Stupp
>Priority: Minor
>  Labels: cqlsh
> Fix For: 3.0.x
>
>
> I found this while addressing CASSANDRA-10289.
> In cqlsh, if the user enters {{USE ks}}, but there is no keyspace named 
> {{ks}}, the prompt will read {{cqlsh:ks>}}. It should just read {{cqlsh>}}, 
> since the underlying session did not actually switch to use {{ks}}.
> I believe the bug is in cqlsh and not, e.g., the driver, because the 
> statement, as expected, raises an {{InvalidRequest}} error.
> The behavior shows in a test in the cqlshlib nosetests here:
> https://github.com/apache/cassandra/blob/03f556ffa8718754fe4eb329af2002d83ffc7147/pylib/cqlshlib/test/test_cqlsh_output.py#L545
> An example failure on CassCI is here:
> http://cassci.datastax.com/job/scratch_mambocab-fix_cqlsh/11/testReport/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/test_prompt/
> You can also reproduce it trivially in ccm, or however you choose to run 
> clusters locally:
> {code}
> ccm create cqlsh -v git:trunk -n 1 ; ccm start --wait-for-binary-proto ; ccm 
> node1 cqlsh
> http://git-wip-us.apache.org/repos/asf/cassandra.git git:trunk
> Fetching Cassandra updates...
> Current cluster is now: cqlsh
> Connected to cqlsh at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.0-beta2-SNAPSHOT | CQL spec 3.3.1 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> use nonexistentkeyspace;
> InvalidRequest: code=2200 [Invalid query] message="Keyspace 
> 'nonexistentkeyspace' does not exist"
> cqlsh:nonexistentkeyspace> 
> {code}
> That last line should read {{cqlsh>}} instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10369) cqlsh prompt includes name of keyspace after failed `use` statement

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-10369:
-
Fix Version/s: 2.2.x

> cqlsh prompt includes name of keyspace after failed `use` statement
> ---
>
> Key: CASSANDRA-10369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10369
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Robert Stupp
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.2.x, 3.0.x
>
>
> I found this while addressing CASSANDRA-10289.
> In cqlsh, if the user enters {{USE ks}}, but there is no keyspace named 
> {{ks}}, the prompt will read {{cqlsh:ks>}}. It should just read {{cqlsh>}}, 
> since the underlying session did not actually switch to use {{ks}}.
> I believe the bug is in cqlsh and not, e.g., the driver, because the 
> statement, as expected, raises an {{InvalidRequest}} error.
> The behavior shows in a test in the cqlshlib nosetests here:
> https://github.com/apache/cassandra/blob/03f556ffa8718754fe4eb329af2002d83ffc7147/pylib/cqlshlib/test/test_cqlsh_output.py#L545
> An example failure on CassCI is here:
> http://cassci.datastax.com/job/scratch_mambocab-fix_cqlsh/11/testReport/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/test_prompt/
> You can also reproduce it trivially in ccm, or however you choose to run 
> clusters locally:
> {code}
> ccm create cqlsh -v git:trunk -n 1 ; ccm start --wait-for-binary-proto ; ccm 
> node1 cqlsh
> http://git-wip-us.apache.org/repos/asf/cassandra.git git:trunk
> Fetching Cassandra updates...
> Current cluster is now: cqlsh
> Connected to cqlsh at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.0-beta2-SNAPSHOT | CQL spec 3.3.1 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> use nonexistentkeyspace;
> InvalidRequest: code=2200 [Invalid query] message="Keyspace 
> 'nonexistentkeyspace' does not exist"
> cqlsh:nonexistentkeyspace> 
> {code}
> That last line should read {{cqlsh>}} instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: bump release

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 600e1aa17 -> b283eac15


bump release


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c95a7098
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c95a7098
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c95a7098

Branch: refs/heads/trunk
Commit: c95a7098cf77b5b8e96feb7c39aca8fec3a02f9c
Parents: d6e14b3
Author: T Jake Luciani 
Authored: Sat Sep 19 16:03:29 2015 -0400
Committer: T Jake Luciani 
Committed: Sat Sep 19 16:03:29 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c95a7098/build.xml
--
diff --git a/build.xml b/build.xml
index ab9c899..89e3d46 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c95a7098/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 41b87a8..7690e1d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.0.0~rc1) unstable; urgency=medium
+
+  * New release candidate
+
+ -- Jake Luciani   Sat, 19 Sep 2015 16:01:59 -0400
+
 cassandra (3.0.0~beta2) unstable; urgency=medium
 
   * New beta release

cassandra git commit: bump release

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 d6e14b34b -> c95a7098c


bump release


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c95a7098
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c95a7098
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c95a7098

Branch: refs/heads/cassandra-3.0
Commit: c95a7098cf77b5b8e96feb7c39aca8fec3a02f9c
Parents: d6e14b3
Author: T Jake Luciani 
Authored: Sat Sep 19 16:03:29 2015 -0400
Committer: T Jake Luciani 
Committed: Sat Sep 19 16:03:29 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c95a7098/build.xml
--
diff --git a/build.xml b/build.xml
index ab9c899..89e3d46 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c95a7098/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 41b87a8..7690e1d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.0.0~rc1) unstable; urgency=medium
+
+  * New release candidate
+
+ -- Jake Luciani   Sat, 19 Sep 2015 16:01:59 -0400
+
 cassandra (3.0.0~beta2) unstable; urgency=medium
 
   * New beta release

Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.0.0-rc1-tentative [created] c95a7098c

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jake]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b283eac1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b283eac1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b283eac1

Branch: refs/heads/trunk
Commit: b283eac1578f6cc09d77bdb4963f08a50d57c2e6
Parents: 600e1aa c95a709
Author: T Jake Luciani 
Authored: Sat Sep 19 16:04:04 2015 -0400
Committer: T Jake Luciani 
Committed: Sat Sep 19 16:04:04 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

[Albert P Tobey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877311#comment-14877311
 ] 

Albert P Tobey commented on CASSANDRA-7486:
---

Is the picture equally bleak at RF=3?

Do the "2.2 GC" settings include anything other than the defaults from 
cassandra-env.sh? "ps -efw" output is sufficient.

I'd be happy to take a look at the GC logs if they are available.

> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7486) Migrate to G1GC by default

[Jonathan Shook (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877316#comment-14877316
 ] 

Jonathan Shook edited comment on CASSANDRA-7486 at 9/20/15 3:33 AM:


This seems pretty open-and-shut where I would expect a bit more of a nuanced 
test. We've honestly seen G1 be the operative improvement in some cases in the 
field. I'd much prefer to see "needs more analysis" than to see it resolved as 
fixed. CMS will *not* scale with hardware as we go forward. This is not in 
debate.

Ah, nevermind. I see that is what the status is now.



was (Author: jshook):
This seems pretty open-and-shut where I would expect a bit more of a nuanced 
test. We've honestly seen G1 be the operative improvement in some cases in the 
field. I'd much prefer to see "needs more analysis" than to see it resolved as 
fixed. CMS will *not* scale with hardware as we go forward. This is not in 
debate.

An, nevermind. I see that is what the status is now.


> Migrate to G1GC by default
> --
>
> Key: CASSANDRA-7486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Config
>Reporter: Jonathan Ellis
>Assignee: Albert P Tobey
> Fix For: 3.0 alpha 1
>
>
> See 
> http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
>  and https://twitter.com/rbranson/status/482113561431265281
> May want to default 2.1 to G1.
> 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
> Suspect this will help G1 even more than CMS.  (NB this is off by default but 
> needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10378) Make skipping more efficient

[Benedict (JIRA)]

Benedict created CASSANDRA-10378:


 Summary: Make skipping more efficient
 Key: CASSANDRA-10378
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10378
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.x


Following on from the impact of CASSANDRA-10322, we can improve the efficiency 
of our calls to skipping methods. CASSANDRA-10326 is showing our performance to 
be in-and-around the same ballpark except for seeks into the middle of a large 
partition, which suggests (possibly) that the higher density of data we're 
storing may simply be resulting in a more significant CPU burden as we have 
more data to skip over (and since CASSANDRA-10322 improves performance here 
really dramatically, further improvements are likely to be of similar benefit).

I propose doing our best to flatten the skipping of macro data items into as 
few skip invocations as necessary. One way of doing this would be to introduce 
a special {{skipUnsignedVInts(int)}} method, that can efficiently skip a number 
of unsigned vints. Almost the entire body of a cell and row consist of vints 
now, each data component with their own special {{skipX}} method that invokes 
{{readUnsignedVint}}. This would permit more efficient despatch.

We could also potentially avoid the construction of a new {{Columns}} instance 
for each row skip, since all we need is an iterator over the columns, and share 
the temporary space used for storing them, which should further reduce the GC 
burden for skipping many rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Set hints_directory in cassandra_pig.yaml

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 92b813165 -> 4bb5976a7


Set hints_directory in cassandra_pig.yaml


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cffa93c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cffa93c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cffa93c6

Branch: refs/heads/trunk
Commit: cffa93c62958c83948d811a807922f108b3aa714
Parents: 6af4657
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 17:50:06 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 17:50:06 2015 +0100

--
 test/conf/cassandra_pig.yaml | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cffa93c6/test/conf/cassandra_pig.yaml
--
diff --git a/test/conf/cassandra_pig.yaml b/test/conf/cassandra_pig.yaml
index 68615cf..286434b 100644
--- a/test/conf/cassandra_pig.yaml
+++ b/test/conf/cassandra_pig.yaml
@@ -8,6 +8,7 @@ commitlog_sync: batch
 commitlog_sync_batch_window_in_ms: 1.0
 commitlog_segment_size_in_mb: 5
 commitlog_directory: build/test/cassandra/commitlog
+hints_directory: build/test/cassandra/hints
 partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 listen_address: 127.0.0.1
 storage_port: 7010

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4bb5976a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4bb5976a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4bb5976a

Branch: refs/heads/trunk
Commit: 4bb5976a7bd3b68e9ea05610c4b4e174645ad07a
Parents: 92b8131 cffa93c
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 17:50:38 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 17:50:38 2015 +0100

--
 test/conf/cassandra_pig.yaml | 1 +
 1 file changed, 1 insertion(+)
--

cassandra git commit: Set hints_directory in cassandra_pig.yaml

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 6af4657a9 -> cffa93c62


Set hints_directory in cassandra_pig.yaml


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cffa93c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cffa93c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cffa93c6

Branch: refs/heads/cassandra-3.0
Commit: cffa93c62958c83948d811a807922f108b3aa714
Parents: 6af4657
Author: Aleksey Yeschenko 
Authored: Sat Sep 19 17:50:06 2015 +0100
Committer: Aleksey Yeschenko 
Committed: Sat Sep 19 17:50:06 2015 +0100

--
 test/conf/cassandra_pig.yaml | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cffa93c6/test/conf/cassandra_pig.yaml
--
diff --git a/test/conf/cassandra_pig.yaml b/test/conf/cassandra_pig.yaml
index 68615cf..286434b 100644
--- a/test/conf/cassandra_pig.yaml
+++ b/test/conf/cassandra_pig.yaml
@@ -8,6 +8,7 @@ commitlog_sync: batch
 commitlog_sync_batch_window_in_ms: 1.0
 commitlog_segment_size_in_mb: 5
 commitlog_directory: build/test/cassandra/commitlog
+hints_directory: build/test/cassandra/hints
 partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 listen_address: 127.0.0.1
 storage_port: 7010