[jira] [Updated] (CASSANDRA-9875) Rebuild from targeted replica

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-9875:
---
Summary: Rebuild from targeted replica  (was: Rebuild with start and end 
token and from targeted replica)

> Rebuild from targeted replica
> -
>
> Key: CASSANDRA-9875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9875
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
>  Labels: lhf
> Fix For: 3.x
>
> Attachments: 9875-trunk.txt
>
>
> Nodetool rebuild command will rebuild all the token ranges handled by the 
> endpoint. Sometimes we want to rebuild only a certain token range. We should 
> add this ability to rebuild command. We should also add the ability to stream 
> from a given replica.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9875) Rebuild from targeted replica

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-9875:
---
Fix Version/s: 3.x
   Status: Patch Available  (was: Open)

> Rebuild from targeted replica
> -
>
> Key: CASSANDRA-9875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9875
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Geoffrey Yu
>Priority: Minor
>  Labels: lhf
> Fix For: 3.x
>
> Attachments: 9875-trunk.txt
>
>
> Nodetool rebuild command will rebuild all the token ranges handled by the 
> endpoint. Sometimes we want to rebuild only a certain token range. We should 
> add this ability to rebuild command. We should also add the ability to stream 
> from a given replica.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-9875) Rebuild from targeted replica

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu reassigned CASSANDRA-9875:
--

Assignee: Geoffrey Yu

> Rebuild from targeted replica
> -
>
> Key: CASSANDRA-9875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9875
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Geoffrey Yu
>Priority: Minor
>  Labels: lhf
> Fix For: 3.x
>
> Attachments: 9875-trunk.txt
>
>
> Nodetool rebuild command will rebuild all the token ranges handled by the 
> endpoint. Sometimes we want to rebuild only a certain token range. We should 
> add this ability to rebuild command. We should also add the ability to stream 
> from a given replica.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9875) Rebuild with start and end token and from targeted replica

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-9875:
---
Attachment: 9875-trunk.txt

Since CASSANDRA-10406 already implements the ability to specify ranges for 
{{nodetool rebuild}}, I attached a patch to add the ability to specify specific 
sources to stream from for the rebuild (which is the other improvement this 
ticket mentions).

*Usage:*

{{nodetool rebuild --keyspace  --tokens  --sources }}

The implementation in this ticket requires that if {{-- sources}} is used, a 
source must be specified for every single token range provided using {{-- 
tokens}}.

I also added in some code to validate the inputted ranges to make sure that the 
current node owns all of them.

> Rebuild with start and end token and from targeted replica
> --
>
> Key: CASSANDRA-9875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9875
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
>  Labels: lhf
> Attachments: 9875-trunk.txt
>
>
> Nodetool rebuild command will rebuild all the token ranges handled by the 
> endpoint. Sometimes we want to rebuild only a certain token range. We should 
> add this ability to rebuild command. We should also add the ability to stream 
> from a given replica.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12311) Propagate TombstoneOverwhelmingException to the client

[Geoffrey Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414589#comment-15414589
 ] 

Geoffrey Yu commented on CASSANDRA-12311:
-

Thanks, that sounds great!

> Propagate TombstoneOverwhelmingException to the client
> --
>
> Key: CASSANDRA-12311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12311
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
>  Labels: client-impacting, doc-impacting
> Fix For: 4.x
>
> Attachments: 12311-dtest.txt, 12311-trunk-v2.txt, 12311-trunk-v3.txt, 
> 12311-trunk-v4.txt, 12311-trunk-v5.txt, 12311-trunk.txt
>
>
> Right now if a data node fails to perform a read because it ran into a 
> {{TombstoneOverwhelmingException}}, it only responds back to the coordinator 
> node with a generic failure. Under this scheme, the coordinator won't be able 
> to know exactly why the request failed and subsequently the client only gets 
> a generic {{ReadFailureException}}. It would be useful to inform the client 
> that their read failed because we read too many tombstones. We should have 
> the data nodes reply with a failure type so the coordinator can pass this 
> information to the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9876) One way targeted repair

[Geoffrey Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414586#comment-15414586
 ] 

Geoffrey Yu edited comment on CASSANDRA-9876 at 8/10/16 1:51 AM:
-

Thanks for the quick review! I’ve attached a new patch that addresses your 
comments, with the exception of one of them for which I wanted to get some more 
feedback first.

I also attached a patch that adds one dtest to test the pull repair. It works 
nearly identically to the token range repair with the exception that it asserts 
that one of the nodes only sends data and the other only receives.

{quote}
I don't think it's necessary to make specifying --start-token and --end-token 
mandatory, since if that is not specified it will just pull repair all common 
ranges between specified hosts.
{quote}

The reason why I added in the check for a token range was that the repair code 
as it is now doesn’t actually add only the common ranges between the specified 
hosts. I wasn’t sure if this is was the intended behavior or a bug.

To replicate the issue, just create a 3 node cluster, add a keyspace with 
replication factor 2, and run a regular repair through nodetool on that 
keyspace with exactly two nodes specified.

The reason it happens is that if no ranges are specified, the repair will [add 
all ranges on the local 
node|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3137].
 Then when we hit {{RepairRunnable}}, we try to [find a list of neighbors for 
each 
range|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairRunnable.java#L160-L162].

The problem here is that it isn’t always true that every range the local node 
owns is also owned by the remote node we specified through the nodetool 
command. Because of this the [check 
here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ActiveRepairService.java#L246-L251]
 may result in an exception being thrown, which aborts the repair.

If this is intended behavior, then forcing the user to specify a token range 
that is common between the nodes prevents that exception from being thrown. 
Otherwise the error message, “Repair requires at least two endpoints that are 
neighbours before it can continue” can be confusing to the operator since the 
two specified nodes may actually share a common range. What do you think?


was (Author: geoffxy):
Thanks for the quick review! I’ve attached a new patch that addresses your 
comments, with the exception of one of them for which I wanted to get some more 
feedback first.

I also attached a patch that adds one dtest to test the pull repair. It works 
nearly identically to the token range repair with the exception that it asserts 
that one of the nodes only sends data and the other only receives.

{quote}
I don't think it's necessary to make specifying --start-token and --end-token 
mandatory, since if that is not specified it will just pull repair all common 
ranges between specified hosts.
{quote}

The reason why I added in the check for a token range was that the repair code 
as it is now doesn’t actually add only the common ranges between the specified 
hosts. I wasn’t sure if this is was the intended behavior or a bug.

To replicate the issue, just create a 3 node cluster, add a keyspace with 
replication factor 2, and run a regular repair through nodetool on that 
keyspace with exactly two nodes specified.

The reason it happens is that if no ranges are specified, the repair will [add 
all ranges on the local 
node|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3137].
 Then when we hit {{RepairRunnable}}, we try to [find a list of neighbors for 
each 
range|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairRunnable.java#L160-L162].

The problem here is that it isn’t always true that every range the local node 
owns is also owned by the remote node we specified through the nodetool 
command. In the example above, only one range will be common between any two 
nodes. Because of this the [check 
here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ActiveRepairService.java#L246-L251]
 may result in an exception being thrown, which aborts the repair.

If this is intended behavior, then forcing the user to specify a token range 
that is common between the nodes prevents that exception from being thrown. 
Otherwise the error message, “Repair requires at least two endpoints that are 
neighbours before it can continue” can be confusing to the operator since the 
two specified nodes may actually share a common range. What do you think?

> One way targeted repair
> ---
>
> Key: CASSANDRA-9876
> URL: 

[jira] [Comment Edited] (CASSANDRA-9876) One way targeted repair

[Geoffrey Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414586#comment-15414586
 ] 

Geoffrey Yu edited comment on CASSANDRA-9876 at 8/10/16 1:50 AM:
-

Thanks for the quick review! I’ve attached a new patch that addresses your 
comments, with the exception of one of them for which I wanted to get some more 
feedback first.

I also attached a patch that adds one dtest to test the pull repair. It works 
nearly identically to the token range repair with the exception that it asserts 
that one of the nodes only sends data and the other only receives.

{quote}
I don't think it's necessary to make specifying --start-token and --end-token 
mandatory, since if that is not specified it will just pull repair all common 
ranges between specified hosts.
{quote}

The reason why I added in the check for a token range was that the repair code 
as it is now doesn’t actually add only the common ranges between the specified 
hosts. I wasn’t sure if this is was the intended behavior or a bug.

To replicate the issue, just create a 3 node cluster, add a keyspace with 
replication factor 2, and run a regular repair through nodetool on that 
keyspace with exactly two nodes specified.

The reason it happens is that if no ranges are specified, the repair will [add 
all ranges on the local 
node|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3137].
 Then when we hit {{RepairRunnable}}, we try to [find a list of neighbors for 
each 
range|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairRunnable.java#L160-L162].

The problem here is that it isn’t always true that every range the local node 
owns is also owned by the remote node we specified through the nodetool 
command. In the example above, only one range will be common between any two 
nodes. Because of this the [check 
here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ActiveRepairService.java#L246-L251]
 may result in an exception being thrown, which aborts the repair.

If this is intended behavior, then forcing the user to specify a token range 
that is common between the nodes prevents that exception from being thrown. 
Otherwise the error message, “Repair requires at least two endpoints that are 
neighbours before it can continue” can be confusing to the operator since the 
two specified nodes may actually share a common range. What do you think?


was (Author: geoffxy):
Thanks for the quick review! I’ve attached a new patch that addresses your 
comments, with the exception of one of them for which I wanted to get some more 
feedback first.

I also attached a patch that adds one dtest to test the pull repair. It works 
nearly identically to the token range repair with the exception that it asserts 
that one of the nodes only sends data and the other only receives.

{quote}
I don't think it's necessary to make specifying --start-token and --end-token 
mandatory, since if that is not specified it will just pull repair all common 
ranges between specified hosts.
{quote}

The reason why I added in the check for a token range was that the repair code 
as it is now doesn’t actually add only the common ranges between the specified 
hosts. I wasn’t sure if this is was the intended behavior or a bug.

To replicate the issue, just create a 3 node cluster, add a keyspace with 
replication factor 2, and run a regular repair through nodetool on that 
keyspace with exactly two nodes specified.

The reason it happens is that if no ranges are specified, the repair will [add 
all ranges on the local 
node|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3137].
 Then when we hit {{RepairRunnable}}, we try to find a list of neighbors for 
each range 
(https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairRunnable.java#L160-L162).

The problem here is that it isn’t always true that every range the local node 
owns is also owned by the remote node we specified through the nodetool 
command. In the example above, only one range will be common between any two 
nodes. Because of this the [check 
here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ActiveRepairService.java#L246-L251]
 may result in an exception being thrown, which aborts the repair.

If this is intended behavior, then forcing the user to specify a token range 
that is common between the nodes prevents that exception from being thrown. 
Otherwise the error message, “Repair requires at least two endpoints that are 
neighbours before it can continue” can be confusing to the operator since the 
two specified nodes may actually share a common range. What do you think?

> One way targeted repair
> ---
>
>  

[jira] [Updated] (CASSANDRA-9876) One way targeted repair

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-9876:
---
Status: Awaiting Feedback  (was: Open)

> One way targeted repair
> ---
>
> Key: CASSANDRA-9876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9876
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 9876-dtest-master.txt, 9876-trunk-v2.txt, 9876-trunk.txt
>
>
> Many applications use C* by writing to one local DC. The other DC is used 
> when the local DC is unavailable. When the local DC becomes available, we 
> want to run a targeted repair b/w one endpoint from each DC to minimize the 
> data transfer over WAN.  In this case, it will be helpful to do a one way 
> repair in which data will only be streamed from other DC to local DC instead 
> of streaming the data both ways. This will further minimize the traffic over 
> WAN. This feature should only be supported if a targeted repair is run 
> involving 2 hosts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9876) One way targeted repair

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-9876:
---
Attachment: 9876-dtest-master.txt
9876-trunk-v2.txt

Thanks for the quick review! I’ve attached a new patch that addresses your 
comments, with the exception of one of them for which I wanted to get some more 
feedback first.

I also attached a patch that adds one dtest to test the pull repair. It works 
nearly identically to the token range repair with the exception that it asserts 
that one of the nodes only sends data and the other only receives.

{quote}
I don't think it's necessary to make specifying --start-token and --end-token 
mandatory, since if that is not specified it will just pull repair all common 
ranges between specified hosts.
{quote}

The reason why I added in the check for a token range was that the repair code 
as it is now doesn’t actually add only the common ranges between the specified 
hosts. I wasn’t sure if this is was the intended behavior or a bug.

To replicate the issue, just create a 3 node cluster, add a keyspace with 
replication factor 2, and run a regular repair through nodetool on that 
keyspace with exactly two nodes specified.

The reason it happens is that if no ranges are specified, the repair will [add 
all ranges on the local 
node|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3137].
 Then when we hit {{RepairRunnable}}, we try to find a list of neighbors for 
each range 
(https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairRunnable.java#L160-L162).

The problem here is that it isn’t always true that every range the local node 
owns is also owned by the remote node we specified through the nodetool 
command. In the example above, only one range will be common between any two 
nodes. Because of this the [check 
here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ActiveRepairService.java#L246-L251]
 may result in an exception being thrown, which aborts the repair.

If this is intended behavior, then forcing the user to specify a token range 
that is common between the nodes prevents that exception from being thrown. 
Otherwise the error message, “Repair requires at least two endpoints that are 
neighbours before it can continue” can be confusing to the operator since the 
two specified nodes may actually share a common range. What do you think?

> One way targeted repair
> ---
>
> Key: CASSANDRA-9876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9876
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 9876-dtest-master.txt, 9876-trunk-v2.txt, 9876-trunk.txt
>
>
> Many applications use C* by writing to one local DC. The other DC is used 
> when the local DC is unavailable. When the local DC becomes available, we 
> want to run a targeted repair b/w one endpoint from each DC to minimize the 
> data transfer over WAN.  In this case, it will be helpful to do a one way 
> repair in which data will only be streamed from other DC to local DC instead 
> of streaming the data both ways. This will further minimize the traffic over 
> WAN. This feature should only be supported if a targeted repair is run 
> involving 2 hosts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11701) [windows] dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414568#comment-15414568
 ] 

Stefania commented on CASSANDRA-11701:
--

Thanks for your input, agreed on limiting this patch to 2.2+.

> [windows] dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows
> -
>
> Key: CASSANDRA-11701
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11701
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Stefania
>  Labels: dtest, windows
>
> looks to be an assertion problem, so could be test or cassandra related:
> e.g.:
> {noformat}
> 1 != 331
> {noformat}
> http://cassci.datastax.com/job/trunk_dtest_win32/404/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_with_skip_and_max_rows
> Failed on CassCI build trunk_dtest_win32 #404



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12407) dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12407:
-
Component/s: Testing

> dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test
> 
>
> Key: CASSANDRA-12407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12407
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/381/testReport/cql_tracing_test/TestCqlTracing/tracing_simple_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 102, in 
> tracing_simple_test
> self.trace(session)
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 74, in 
> trace
> self.assertIn('/127.0.0.1', out)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> '\'/127.0.0.1\' not found in "Consistency level set to ALL.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12407) dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12407:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Pull request merged, test will be skipped in 2.1.

> dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test
> 
>
> Key: CASSANDRA-12407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12407
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/381/testReport/cql_tracing_test/TestCqlTracing/tracing_simple_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 102, in 
> tracing_simple_test
> self.trace(session)
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 74, in 
> trace
> self.assertIn('/127.0.0.1', out)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> '\'/127.0.0.1\' not found in "Consistency level set to ALL.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12212:

Reviewer: Aleksey Yeschenko

> system.compactions_in_progress needs to be used on first upgrade to 3.0
> ---
>
> Key: CASSANDRA-12212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12212
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jeremiah Jordan
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-7066 removed the system.compactions_in_progress table and replaced 
> it with the new transaction system.  But system.compactions_in_progress needs 
> to be consulted for the first startup after upgrading from 2.1 to 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12406) dtest failure in pushed_notifications_test.TestPushedNotifications.move_single_node_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12406:

Assignee: Paulo Motta

> dtest failure in 
> pushed_notifications_test.TestPushedNotifications.move_single_node_test
> 
>
> Key: CASSANDRA-12406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12406
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Paulo Motta
>  Labels: dtest
> Fix For: 2.1.x
>
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/271/testReport/pushed_notifications_test/TestPushedNotifications/move_single_node_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/pushed_notifications_test.py", line 
> 110, in move_single_node_test
> self.assertEquals(1, len(notifications), notifications)
>   File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
>   File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "[{'change_type': u'MOVED_NODE', 'address': ('127.0.0.1', 9042)}, 
> {'change_type': u'NEW_NODE', 'address': ('127.0.0.1', 9042)}]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11611) dtest failure in topology_test.TestTopology.crash_during_decommission_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11611:

Assignee: Paulo Motta

> dtest failure in topology_test.TestTopology.crash_during_decommission_test
> --
>
> Key: CASSANDRA-11611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11611
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Paulo Motta
>  Labels: dtest
> Fix For: 3.x
>
>
> Looks like some kind of streaming error. Example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/382/testReport/topology_test/TestTopology/crash_during_decommission_test
> Failed on CassCI build trunk_dtest_win32 #382
> {code}
> Error Message
> Unexpected error in log, see stdout
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: d:\temp\dtest-ce_wos
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UN  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  162.38 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  222.26 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  336.69 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  360.82 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  240.54 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Decommission failed with exception: Nodetool command 
> 'D:\jenkins\workspace\trunk_dtest_win32\cassandra\bin\nodetool.bat -h 
> localhost -p 7100 decommission' failed; exit status: 2; stderr: error: Stream 
> failed
> -- StackTrace --
> 

[jira] [Updated] (CASSANDRA-12399) dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x.multi_collection_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12399:

Assignee: Benjamin Lerer

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x.multi_collection_test
> 
>
> Key: CASSANDRA-12399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12399
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Benjamin Lerer
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/17/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x/multi_collection_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line 
> 1419, in multi_collection_test
> sortedset([1, 3, 5, 7, 11, 13])
>   File "/home/automaton/cassandra-dtest/assertions.py", line 163, in 
> assert_all
> assert list_res == expected, "Expected {} from {}, but got 
> {}".format(expected, query, list_res)
> "Expected [[[1, 3, 5, 7, 11, 13], OrderedDict([('bar', 3), ('foo', 1), 
> ('foobar', 4)]), SortedSet([1, 3, 5, 7, 11, 13])]] from SELECT L, M, S FROM 
> foo WHERE k = b017f48f-ae67-11e1-9096-005056c8, but got [[[1, 3, 5, 1, 3, 
> 5, 7, 11, 13], OrderedMapSerializedKey([(u'bar', 3), (u'foo', 1), (u'foobar', 
> 4)]), SortedSet([1, 3, 5, 7, 11, 13])]]
> {code}
> Related failure:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/17/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/multi_collection_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12401) dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.multi_list_set_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12401:

Assignee: Benjamin Lerer

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.multi_list_set_test
> 
>
> Key: CASSANDRA-12401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12401
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Benjamin Lerer
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/17/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/multi_list_set_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line 
> 2289, in multi_list_set_test
> assert_one(cursor, "SELECT l1, l2 FROM test WHERE k = 0", [[1, 24, 3], 
> [4, 42, 6]])
>   File "/home/automaton/cassandra-dtest/assertions.py", line 124, in 
> assert_one
> assert list_res == [expected], "Expected {} from {}, but got 
> {}".format([expected], query, list_res)
> "Expected [[[1, 24, 3], [4, 42, 6]]] from SELECT l1, l2 FROM test WHERE k = 
> 0, but got [[[1, 24, 3, 1, 2, 3], [4, 42, 6, 4, 5, 6]]]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12362) dtest failure in upgrade_tests.paging_test.TestPagingDatasetChangesNodes2RF1_Upgrade_current_3_x_To_indev_3_x.test_row_TTL_expiry_during_paging

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12362:

Assignee: Jason Brown

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDatasetChangesNodes2RF1_Upgrade_current_3_x_To_indev_3_x.test_row_TTL_expiry_during_paging
> ---
>
> Key: CASSANDRA-12362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12362
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Jason Brown
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_upgrade/5/testReport/upgrade_tests.paging_test/TestPagingDatasetChangesNodes2RF1_Upgrade_current_3_x_To_indev_3_x/test_row_TTL_expiry_during_paging
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/upgrade_tests/paging_test.py", line 
> 1217, in test_row_TTL_expiry_during_paging
> self.assertEqual(pf.pagecount(), 3)
>   File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
>   File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "2 != 3
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12383) dtest failure in batch_test.TestBatch.logged_batch_doesnt_throw_uae_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12383:

Assignee: Alex Petrov

> dtest failure in batch_test.TestBatch.logged_batch_doesnt_throw_uae_test
> 
>
> Key: CASSANDRA-12383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12383
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Craig Kodman
>Assignee: Alex Petrov
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/282/testReport/batch_test/TestBatch/logged_batch_doesnt_throw_uae_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11611) dtest failure in topology_test.TestTopology.crash_during_decommission_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11611:

Issue Type: Bug  (was: Test)

> dtest failure in topology_test.TestTopology.crash_during_decommission_test
> --
>
> Key: CASSANDRA-11611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11611
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.x
>
>
> Looks like some kind of streaming error. Example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/382/testReport/topology_test/TestTopology/crash_during_decommission_test
> Failed on CassCI build trunk_dtest_win32 #382
> {code}
> Error Message
> Unexpected error in log, see stdout
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: d:\temp\dtest-ce_wos
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UN  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  162.38 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  222.26 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  336.69 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  360.82 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  240.54 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Decommission failed with exception: Nodetool command 
> 'D:\jenkins\workspace\trunk_dtest_win32\cassandra\bin\nodetool.bat -h 
> localhost -p 7100 decommission' failed; exit status: 2; stderr: error: Stream 
> failed
> -- StackTrace --
> 

[jira] [Updated] (CASSANDRA-12354) dtest failure in upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_2_2_x.bug_5732_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12354:

Assignee: Eduard Tudenhoefner

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_2_2_x.bug_5732_test
> 
>
> Key: CASSANDRA-12354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12354
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Craig Kodman
>Assignee: Eduard Tudenhoefner
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_upgrade/7/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_2_2_x/bug_5732_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11611) dtest failure in topology_test.TestTopology.crash_during_decommission_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11611:

Assignee: (was: DS Test Eng)

> dtest failure in topology_test.TestTopology.crash_during_decommission_test
> --
>
> Key: CASSANDRA-11611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11611
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>  Labels: dtest
> Fix For: 3.x
>
>
> Looks like some kind of streaming error. Example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/382/testReport/topology_test/TestTopology/crash_during_decommission_test
> Failed on CassCI build trunk_dtest_win32 #382
> {code}
> Error Message
> Unexpected error in log, see stdout
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: d:\temp\dtest-ce_wos
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UN  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  162.38 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  222.26 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  336.69 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  360.82 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  240.54 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Decommission failed with exception: Nodetool command 
> 'D:\jenkins\workspace\trunk_dtest_win32\cassandra\bin\nodetool.bat -h 
> localhost -p 7100 decommission' failed; exit status: 2; stderr: error: Stream 
> failed
> -- StackTrace --
> 

[jira] [Updated] (CASSANDRA-11611) dtest failure in topology_test.TestTopology.crash_during_decommission_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11611:

Fix Version/s: 3.x

> dtest failure in topology_test.TestTopology.crash_during_decommission_test
> --
>
> Key: CASSANDRA-11611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11611
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>  Labels: dtest
> Fix For: 3.x
>
>
> Looks like some kind of streaming error. Example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/382/testReport/topology_test/TestTopology/crash_during_decommission_test
> Failed on CassCI build trunk_dtest_win32 #382
> {code}
> Error Message
> Unexpected error in log, see stdout
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: d:\temp\dtest-ce_wos
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UN  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  162.38 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  222.26 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  336.69 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  360.82 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  240.54 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Decommission failed with exception: Nodetool command 
> 'D:\jenkins\workspace\trunk_dtest_win32\cassandra\bin\nodetool.bat -h 
> localhost -p 7100 decommission' failed; exit status: 2; stderr: error: Stream 
> failed
> -- StackTrace --
> org.apache.cassandra.streaming.StreamException: 

[jira] [Commented] (CASSANDRA-11611) dtest failure in topology_test.TestTopology.crash_during_decommission_test

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414517#comment-15414517
 ] 

Philip Thompson commented on CASSANDRA-11611:
-

See 
http://cassci.datastax.com/job/cassandra-3.9_dtest/lastCompletedBuild/testReport/topology_test/TestTopology/crash_during_decommission_test/

for a linux failure

> dtest failure in topology_test.TestTopology.crash_during_decommission_test
> --
>
> Key: CASSANDRA-11611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11611
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.x
>
>
> Looks like some kind of streaming error. Example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/382/testReport/topology_test/TestTopology/crash_during_decommission_test
> Failed on CassCI build trunk_dtest_win32 #382
> {code}
> Error Message
> Unexpected error in log, see stdout
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: d:\temp\dtest-ce_wos
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UN  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  162.38 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  222.26 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  336.69 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  360.82 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  240.54 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Decommission failed with exception: Nodetool command 
> 

[jira] [Updated] (CASSANDRA-11611) dtest failure in topology_test.TestTopology.crash_during_decommission_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11611:

Labels: dtest  (was: dtest windows)

> dtest failure in topology_test.TestTopology.crash_during_decommission_test
> --
>
> Key: CASSANDRA-11611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11611
> Project: Cassandra
>  Issue Type: Test
>Reporter: Jim Witschey
>Assignee: DS Test Eng
>  Labels: dtest
>
> Looks like some kind of streaming error. Example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/382/testReport/topology_test/TestTopology/crash_during_decommission_test
> Failed on CassCI build trunk_dtest_win32 #382
> {code}
> Error Message
> Unexpected error in log, see stdout
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: d:\temp\dtest-ce_wos
> dtest: DEBUG: Custom init_config not found. Setting defaults.
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UN  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  162.38 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  98.73 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  222.26 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  98.71 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  336.69 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  360.82 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Status as reported by node 127.0.0.2
> dtest: DEBUG: Datacenter: datacenter1
> 
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID  
>  Rack
> UL  127.0.0.1  174.2 KiB  32   78.4% 
> b8c55c71-bf3d-462b-8c17-3c88d7ac2284  rack1
> UN  127.0.0.2  240.54 KiB  32   65.9% 
> 71aacf1d-8e2f-44cf-b354-f10c71313ec6  rack1
> UN  127.0.0.3  116.7 KiB  32   55.7% 
> 3a4529a3-dc7f-445c-aec3-94417c920fdf  rack1
> dtest: DEBUG: Restarting node2
> dtest: DEBUG: Decommission failed with exception: Nodetool command 
> 'D:\jenkins\workspace\trunk_dtest_win32\cassandra\bin\nodetool.bat -h 
> localhost -p 7100 decommission' failed; exit status: 2; stderr: error: Stream 
> failed
> -- StackTrace --
> 

[jira] [Updated] (CASSANDRA-12385) Disk failure policy should not be invoked on out of space

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12385:

Reviewer: Jason Brown

> Disk failure policy should not be invoked on out of space
> -
>
> Key: CASSANDRA-12385
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12385
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Minor
> Attachments: CASSANDRA-12385_3.0.txt
>
>
> If a node fills up temporarily due to compaction the disk failure policy may 
> be invoked. We
> use stop, so the node will be disabled. This leaves the node down even though 
> it recovers from this
> failure by aborting the compaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12421) Add the option to only gossip manual severity, not severity from IOWait.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12421:

Reviewer: Aleksey Yeschenko

> Add the option to only gossip manual severity, not severity from IOWait.
> 
>
> Key: CASSANDRA-12421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12421
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 2.1.14
>
> Attachments: 0001-Add-the-option-to-ignore-load-severity.patch
>
>
> Similar to CASSANDRA-11737, but I'd like to still respect the manual 
> severity, and ignore the severity calculated from IOWait/Compaction, since in 
> general disk throughput is not a problem for flash card.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12349) Adding some new features to cqlsh

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12349:

Reviewer: Marcus Eriksson

> Adding some new features to cqlsh
> -
>
> Key: CASSANDRA-12349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12349
> Project: Cassandra
>  Issue Type: New Feature
> Environment: All
>Reporter: vin01
>Priority: Minor
>  Labels: CQLSH
>
> I will like to have following features in in cqlsh, I have made a patch to 
> enable them as well.
> 1. Aliases.
> 2. Safe mode (prompt on delete,update,truncate,drop if safe_mode is true).
> 3. Press q to exit.
> Its also shared here -> 
> https://github.com/vineet01/cassandra/blob/trunk/new_features.txt
> Example for aliases :-
> cassandra@cqlsh> show 
>  ;  ALIASES  HOST SESSION  VERSION  
> cassandra@cqlsh> show ALIASES ;
> Aliases :> {'dk': 'desc keyspaces;', 'sl': 'select * from'}
> now if you type dk and press  it will auto complete it to "desc 
> keyspace".
> Adding an alias from shell :-
> cassandra@cqlsh> alias slu=select * from login.user ;
> Alias added successfully - sle:select * login.user ;
> cassandra@cqlsh> show ALIASES ;
> Aliases :> {'slu': 'select * from login.user ;', 'dk': 'desc keyspaces;', 
> 'sl': 'select * from'}
> cassandra@cqlsh> sle
> Expanded alias to> select * from login.user ;
>  username | blacklisted | lastlogin | password   
> Adding an alias directly in file :-
> aliases will be kept in same cqlshrc file.
> [aliases]
> dk = desc keyspaces;
> sl = select * from
> sle = select * from login.user ;
> now if we type just "sle" it will autocomplete rest of it and show next 
> options.
> Example of safe mode :-
> cassandra@cqlsh> truncate login.user ;
> Are you sure you want to do this? (y/n) > n
> Not performing any action.
> cassandra@cqlsh> updatee login.user set password=null;
> Are you sure you want to do this? (y/n) > 
> Not performing any action.
> Initial commit :- 
> https://github.com/vineet01/cassandra/commit/0bfce2ccfc610021a74a1f82ed24aa63e1b72fec
> Current version :- 
> https://github.com/vineet01/cassandra/blob/trunk/bin/cqlsh.py
> Please review and suggest any improvements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12384) Include info about sstable on "Compacting large row” message

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12384:

Reviewer: Jason Brown

> Include info about sstable on "Compacting large row” message
> 
>
> Key: CASSANDRA-12384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12384
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Trivial
> Attachments: CASSANDRA-12384_3.0.txt, CASSANDRA-12384_trunk.txt, 
> Dtest.txt
>
>
> On a message like this one, it would be helpful to understand which sstable 
> this large row is going in
> Compacting large row abc/xyz:38956kjhawf (xyz bytes) incrementally



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12268) Make MV Index creation robust for wide referent rows

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12268:

Reviewer: T Jake Luciani

> Make MV Index creation robust for wide referent rows
> 
>
> Key: CASSANDRA-12268
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12268
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jonathan Shook
>Assignee: Carl Yeksigian
> Fix For: 3.0.x, 3.x
>
> Attachments: 12268.py
>
>
> When creating an index for a materialized view for extant data, heap pressure 
> is very dependent on the cardinality of of rows associated with each index 
> value. With the way that per-index value rows are created within the index, 
> this can cause unbounded heap pressure, which can cause OOM. This appears to 
> be a side-effect of how each index row is applied atomically as with batches.
> The commit logs can accumulate enough during the process to prevent the node 
> from being restarted. Given that this occurs during global index creation, 
> this can happen on multiple nodes, making stable recovery of a node set 
> difficult, as co-replicas become unavailable to assist in back-filling data 
> from commitlogs.
> While it is understandable that you want to avoid having relatively wide rows 
>  even in materialized views, this represents a particularly difficult 
> scenario for triage.
> The basic recommendation for improving this is to sub-group the index 
> creation into smaller chunks internally, providing a maximal bound against 
> the heap pressure when it is needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12331) Unreleased Resource: Sockets

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12331:

Reviewer: Yuki Morishita

> Unreleased Resource: Sockets
> 
>
> Key: CASSANDRA-12331
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12331
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>  Labels: easyfix, newbie, patch
> Attachments: 12331-3.0.txt
>
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5 
> of the Cassandra source code. The analysis included an automated analysis 
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools 
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> Sockets are low level resources that must be explicitly released so 
> subsequent callers will have access to previously used sockets. In the file 
> RMIServerSocketFactoryImpl.java on lines 15-16 a socket is acquired and 
> eventually returned to the caller on line 18.
> If an exception is thrown by the code on line 17 the socket acquired on lines 
> 15-16 will not be released for subsequent reuse.
> RMIServerSocketFactoryImpl.java, lines 13-19:
> {code:java}
> 13 public ServerSocket createServerSocket(final int pPort) throws IOException
> 14 {
> 15 ServerSocket socket = ServerSocketFactory.getDefault()
> 16  .createServerSocket(pPort, 0, 
> InetAddress.getLoopbackAddress());
> 17 socket.setReuseAddress(true);
> 18 return socket;
> 19 }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12256) Properly respect the request timeouts

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12256:

Reviewer: Tyler Hobbs

> Properly respect the request timeouts
> -
>
> Key: CASSANDRA-12256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12256
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Geoffrey Yu
> Fix For: 3.x
>
> Attachments: 12256-trunk.txt
>
>
> We have a number of {{request_timeout_*}} option, that probably every user 
> expect to be an upper bound on how long the coordinator will wait before 
> timeouting a request, but it's actually not always the case, especially for 
> read requests.
> I believe we don't respect those timeout properly in at least the following 
> cases:
> * On a digest mismatch: in that case, we reset the timeout for the data 
> query, which means the overall query might take up to twice the configured 
> timeout before timeouting.
> * On a range query: the timeout is reset for every sub-range that is queried. 
> With many nodes and vnodes, a range query could span tons of sub-range and so 
> a range query could take pretty much arbitrary long before actually 
> timeouting for the user.
> * On short reads: we also reset the timeout for every short reads "retries".
> It's also worth noting that even outside those, the timeouts don't take most 
> of the processing done by the coordinator (query parsing and CQL handling for 
> instance) into account.
> Now, in all fairness, the reason this is this way is that the timeout 
> currently are *not* timeout for the full user request, but rather how long a 
> coordinator should wait on any given replica for any given internal query 
> before giving up. *However*, I'm pretty sure this is not what user 
> intuitively expect and want, *especially* in the context of CASSANDRA-2848 
> where the goal is explicitely to have an upper bound on the query from the 
> user point of view.
> So I'm suggesting we change how those timeouts are handled to really be 
> timeouts on the whole user query.
> And by that I basically just mean that we'd mark the start of each query as 
> soon as possible in the processing, and use that starting time as base in 
> {{ReadCallback.await}} and {{AbstractWriteResponseHandler.get()}}. It won't 
> be perfect in the sense that we'll still only possibly timeout during 
> "blocking" operations, so typically if parsing a query takes more than your 
> timeout, you still won't timeout until that query is sent, but I think that's 
> probably fine in practice because 1) if you timeouts are small enough that 
> this matter, you're probably doing it wrong and 2) we can totally improve on 
> that later if needs be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12251) dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x.whole_list_conditional_test

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12251:

Reviewer: Joel Knighton

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x.whole_list_conditional_test
> --
>
> Key: CASSANDRA-12251
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12251
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Alex Petrov
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x/whole_list_conditional_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> Relevant error in logs is
> {code}
> Unexpected error in node1 log, error: 
> ERROR [InternalResponseStage:2] 2016-07-20 04:58:45,876 
> CassandraDaemon.java:217 - Exception in thread 
> Thread[InternalResponseStage:2,5,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut 
> down
>   at 
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) 
> ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) 
> ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:165)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
>  ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:842)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.switchMemtableIfCurrent(ColumnFamilyStore.java:822)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:891)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$1(SchemaKeyspace.java:279)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$200/1129213153.accept(Unknown
>  Source) ~[na:na]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:279) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1271)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1253)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationTask$1.response(MigrationTask.java:92) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:53)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_51]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_51]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> {code}
> This is on a mixed 3.0.8, 3.8-tentative cluster



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12100:

Reviewer: Marcus Eriksson

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 3.0.x
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11701) [windows] dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11701:

Reviewer: Paulo Motta

> [windows] dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows
> -
>
> Key: CASSANDRA-11701
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11701
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Stefania
>  Labels: dtest, windows
>
> looks to be an assertion problem, so could be test or cassandra related:
> e.g.:
> {noformat}
> 1 != 331
> {noformat}
> http://cassci.datastax.com/job/trunk_dtest_win32/404/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_with_skip_and_max_rows
> Failed on CassCI build trunk_dtest_win32 #404



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11635) test-clientutil-jar unit test fails

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11635:

Reviewer: Tyler Hobbs

> test-clientutil-jar unit test fails
> ---
>
> Key: CASSANDRA-11635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11635
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Sylvain Lebresne
>  Labels: unittest
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> {noformat}
> test-clientutil-jar:
> [junit] Testsuite: org.apache.cassandra.serializers.ClientUtilsTest
> [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
> 0.314 sec
> [junit] 
> [junit] Testcase: test(org.apache.cassandra.serializers.ClientUtilsTest): 
>   Caused an ERROR
> [junit] org/apache/cassandra/utils/SigarLibrary
> [junit] java.lang.NoClassDefFoundError: 
> org/apache/cassandra/utils/SigarLibrary
> [junit] at org.apache.cassandra.utils.UUIDGen.hash(UUIDGen.java:328)
> [junit] at 
> org.apache.cassandra.utils.UUIDGen.makeNode(UUIDGen.java:307)
> [junit] at 
> org.apache.cassandra.utils.UUIDGen.makeClockSeqAndNode(UUIDGen.java:256)
> [junit] at 
> org.apache.cassandra.utils.UUIDGen.(UUIDGen.java:39)
> [junit] at 
> org.apache.cassandra.serializers.ClientUtilsTest.test(ClientUtilsTest.java:56)
> [junit] Caused by: java.lang.ClassNotFoundException: 
> org.apache.cassandra.utils.SigarLibrary
> [junit] at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> [junit] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> [junit] at 
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> [junit] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> [junit] 
> [junit] 
> [junit] Test org.apache.cassandra.serializers.ClientUtilsTest FAILED
> BUILD FAILED
> {noformat}
> I'll see if I can find a spot where this passes, but it appears to have been 
> failing for a long time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414505#comment-15414505
 ] 

Russ Hatch commented on CASSANDRA-12249:


There was a dtest hiccup (should now be patched up), so I kicked off another 
build: 
http://cassci.datastax.com/view/Dev/view/knifewine/job/knifewine-upgrade_thobbs_12249-3.8-upgrade/2/

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.8
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11811) dtest failure in snapshot_test.TestArchiveCommitlog.test_archive_commitlog

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414474#comment-15414474
 ] 

Joshua McKenzie commented on CASSANDRA-11811:
-

Removing windows label as 12286 is non-windows repro

> dtest failure in snapshot_test.TestArchiveCommitlog.test_archive_commitlog
> --
>
> Key: CASSANDRA-11811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11811
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Branimir Lambov
>  Labels: dtest
> Fix For: 3.x
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/416/testReport/snapshot_test/TestArchiveCommitlog/test_archive_commitlog
> Failed on CassCI build trunk_dtest_win32 #416
> Relevant error is pasted. This is clearly a test problem. No idea why it only 
> happens on windows, as of yet. Affecting most tests in the 
> TestArchiveCommitlog suite
> {code}
> WARN: Failed to flush node: node1 on shutdown.
> Unexpected error in node1 log, error: 
> ERROR [main] 2016-05-13 21:15:02,701 CassandraDaemon.java:729 - Fatal 
> configuration error
> org.apache.cassandra.exceptions.ConfigurationException: Cannot change the 
> number of tokens from 64 to 32
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1043)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:740)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:625)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:368) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:583)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:712) 
> [main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11811) dtest failure in snapshot_test.TestArchiveCommitlog.test_archive_commitlog

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11811:

Labels: dtest  (was: dtest windows)

> dtest failure in snapshot_test.TestArchiveCommitlog.test_archive_commitlog
> --
>
> Key: CASSANDRA-11811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11811
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Branimir Lambov
>  Labels: dtest
> Fix For: 3.x
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/416/testReport/snapshot_test/TestArchiveCommitlog/test_archive_commitlog
> Failed on CassCI build trunk_dtest_win32 #416
> Relevant error is pasted. This is clearly a test problem. No idea why it only 
> happens on windows, as of yet. Affecting most tests in the 
> TestArchiveCommitlog suite
> {code}
> WARN: Failed to flush node: node1 on shutdown.
> Unexpected error in node1 log, error: 
> ERROR [main] 2016-05-13 21:15:02,701 CassandraDaemon.java:729 - Fatal 
> configuration error
> org.apache.cassandra.exceptions.ConfigurationException: Cannot change the 
> number of tokens from 64 to 32
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1043)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:740)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:625)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:368) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:583)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:712) 
> [main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12283) CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414463#comment-15414463
 ] 

Joshua McKenzie commented on CASSANDRA-12283:
-

I should have clarified: I don't think that code is in any way *clear*, I just 
think it's probably correct for what it does. Maybe.

I'm all for us clearing it up, just would be surprised if that fixed the issue 
is all.

> CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky
> 
>
> Key: CASSANDRA-12283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12283
> Project: Cassandra
>  Issue Type: Test
>Reporter: Joshua McKenzie
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: unittest
>
> Failed 3 of the last 38 runs.
> [Failure|http://cassci.datastax.com/job/cassandra-3.9_testall/lastCompletedBuild/testReport/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerTest/testCompressedCommitLogBackpressure/]
> Details:
> Error Message
> Timeout occurred. Please note the time in the report does not reflect the 
> time until the timeout.
> Stacktrace
> junit.framework.AssertionFailedError: Timeout occurred. Please note the time 
> in the report does not reflect the time until the timeout.
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414390#comment-15414390
 ] 

Russ Hatch commented on CASSANDRA-12249:


No problem -- job is here: 
http://cassci.datastax.com/view/Dev/view/knifewine/job/knifewine-upgrade_thobbs_12249-3.8-upgrade/

The job should queue up soon as build #1. Should take maybe 4-6 hours once 
started (will run on ec2 w/10 nodes to match the last run of the 3.8 upgrade 
job).

If you're using the autojobs setup, feel free to push branches with 'upgrade' 
somewhere in the name, and the autojobs system will build an upgrade job for 
you which you can run on-demand. If you do that, I'd suggest ec2 for now since 
openstack has been a bit more noisy thus far wrt upgrades.

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.8
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12249:

Reviewer: Benjamin Lerer
  Status: Patch Available  (was: Open)

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.8
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12249:

Fix Version/s: (was: 3.x)
   3.8

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.8
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414363#comment-15414363
 ] 

Tyler Hobbs commented on CASSANDRA-12249:
-

The patch specially handles {{PAGED_RANGE}} verbs to convert them to 
{{RANGE_SLICE}} verbs before sending the message.

Patches and pending CI runs:
||branch||testall||dtest||
|[CASSANDRA-12249-3.8|https://github.com/thobbs/cassandra/tree/CASSANDRA-12249-3.8]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12249-3.8-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12249-3.8-dtest]|
|[CASSANDRA-12249-3.9|https://github.com/thobbs/cassandra/tree/CASSANDRA-12249-3.9]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12249-3.9-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12249-3.9-dtest]|
|[CASSANDRA-12249-trunk|https://github.com/thobbs/cassandra/tree/CASSANDRA-12249-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12249-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12249-trunk-dtest]|

[~rhatch] can I get an upgrade test suite run on the CASSANDRA-12249-3.8 
branch?  I've run the test in the title many times locally without a failure, 
but I'd like to see more complete results.

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10992) Hanging streaming sessions

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-10992:
---
   Resolution: Fixed
Fix Version/s: (was: 2.1.x)
   3.9
   3.0.9
   2.2.8
   Status: Resolved  (was: Patch Available)

Thanks, tests look good so +1.
I committed 2.2+ as {{76e3100ffb106cab3cc665404e293c1026e5e65c}} and skipped 
2.1 since this is not critical one.

> Hanging streaming sessions
> --
>
> Key: CASSANDRA-10992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10992
> Project: Cassandra
>  Issue Type: Bug
> Environment: C* 2.1.12, Debian Wheezy
>Reporter: mlowicki
>Assignee: Paulo Motta
> Fix For: 2.2.8, 3.0.9, 3.9
>
> Attachments: apache-cassandra-2.1.12-SNAPSHOT.jar, db1.ams.jstack, 
> db6.analytics.jstack
>
>
> I've started recently running repair using [Cassandra 
> Reaper|https://github.com/spotify/cassandra-reaper]  (built-in {{nodetool 
> repair}} doesn't work for me - CASSANDRA-9935). It behaves fine but I've 
> noticed hanging streaming sessions:
> {code}
> root@db1:~# date
> Sat Jan  9 16:43:00 UTC 2016
> root@db1:~# nt netstats -H | grep total
> Receiving 5 files, 46.59 MB total. Already received 1 files, 11.32 MB 
> total
> Sending 7 files, 46.28 MB total. Already sent 7 files, 46.28 MB total
> Receiving 6 files, 64.15 MB total. Already received 1 files, 12.14 MB 
> total
> Sending 5 files, 61.15 MB total. Already sent 5 files, 61.15 MB total
> Receiving 4 files, 7.75 MB total. Already received 3 files, 7.58 MB 
> total
> Sending 4 files, 4.29 MB total. Already sent 4 files, 4.29 MB total
> Receiving 12 files, 13.79 MB total. Already received 11 files, 7.66 
> MB total
> Sending 5 files, 15.32 MB total. Already sent 5 files, 15.32 MB total
> Receiving 8 files, 20.35 MB total. Already received 1 files, 13.63 MB 
> total
> Sending 38 files, 125.34 MB total. Already sent 38 files, 125.34 MB 
> total
> root@db1:~# date
> Sat Jan  9 17:45:42 UTC 2016
> root@db1:~# nt netstats -H | grep total
> Receiving 5 files, 46.59 MB total. Already received 1 files, 11.32 MB 
> total
> Sending 7 files, 46.28 MB total. Already sent 7 files, 46.28 MB total
> Receiving 6 files, 64.15 MB total. Already received 1 files, 12.14 MB 
> total
> Sending 5 files, 61.15 MB total. Already sent 5 files, 61.15 MB total
> Receiving 4 files, 7.75 MB total. Already received 3 files, 7.58 MB 
> total
> Sending 4 files, 4.29 MB total. Already sent 4 files, 4.29 MB total
> Receiving 12 files, 13.79 MB total. Already received 11 files, 7.66 
> MB total
> Sending 5 files, 15.32 MB total. Already sent 5 files, 15.32 MB total
> Receiving 8 files, 20.35 MB total. Already received 1 files, 13.63 MB 
> total
> Sending 38 files, 125.34 MB total. Already sent 38 files, 125.34 MB 
> total
> {code}
> Such sessions are left even when repair job is long time done (confirmed by 
> checking Reaper's and Cassandra's logs). {{streaming_socket_timeout_in_ms}} 
> in cassandra.yaml is set to default value (360).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[02/10] cassandra git commit: Fix hanging stream session

[yukim]

Fix hanging stream session

by preventing CompressedStreamReader from blocking on IOException.
Also removed retry support from streaming.

Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-10992


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76e3100f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76e3100f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76e3100f

Branch: refs/heads/cassandra-3.0
Commit: 76e3100ffb106cab3cc665404e293c1026e5e65c
Parents: bc9af92
Author: Paulo Motta 
Authored: Thu Jun 23 11:33:54 2016 -0300
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:31:34 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +--
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 -
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 17 +++
 .../compress/CompressedInputStreamTest.java | 49 +---
 11 files changed, 102 insertions(+), 93 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f734476..232203e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.8
+ * Fix hanging stream session (CASSANDRA-10992)
  * Add byteman support for testing (CASSANDRA-12377)
  * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
  * Restore JVM metric export for metric reporters (CASSANDRA-12312)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ede4560..60daee6 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -171,6 +171,10 @@ public class Config
 public volatile Integer compaction_throughput_mb_per_sec = 16;
 public volatile Integer compaction_large_partition_warning_threshold_mb = 
100;
 
+/**
+ * @deprecated retry support removed on CASSANDRA-10992
+ */
+@Deprecated
 public Integer max_streaming_retries = 3;
 
 public volatile Integer stream_throughput_outbound_megabits_per_sec = 200;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index f1acfc4..6e46725 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -957,11 +957,6 @@ public class DatabaseDescriptor
 return conf.cluster_name;
 }
 
-public static int getMaxStreamingRetries()
-{
-return conf.max_streaming_retries;
-}
-
 public static int getStoragePort()
 {
 return Integer.parseInt(System.getProperty("cassandra.storage_port", 
conf.storage_port.toString()));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/streaming/StreamReader.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReader.java 
b/src/java/org/apache/cassandra/streaming/StreamReader.java
index 8789720..c96ea22 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReader.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReader.java
@@ -45,6 +45,7 @@ import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.BytesReadTracker;
 import org.apache.cassandra.utils.Pair;
 
+import static org.apache.cassandra.utils.Throwables.extractIOExceptionCause;
 
 /**
  * StreamReader reads from stream and writes to SSTable.
@@ -137,11 +138,7 @@ public class StreamReader
 e.addSuppressed(e2);
 }
 }
-drain(dis, in.getBytesRead());
-if (e instanceof IOException)
-throw (IOException) e;
-else
-throw 

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8458e4e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8458e4e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8458e4e3

Branch: refs/heads/cassandra-3.9
Commit: 8458e4e30744e8df6fcbe12aec286db047411385
Parents: ee60941 62ef861
Author: Yuki Morishita 
Authored: Tue Aug 9 16:56:29 2016 -0500
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:56:29 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +-
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 +++-
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 14 ++
 .../compression/CompressedInputStreamTest.java  | 52 +---
 11 files changed, 100 insertions(+), 95 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8458e4e3/CHANGES.txt
--
diff --cc CHANGES.txt
index b7bbf72,f613c5f..8fbd774
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,47 -15,6 +22,48 @@@ Merged from 3.0
 to connect with too low of a protocol version (CASSANDRA-11464)
   * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
   * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 +Merged from 2.2:
++ * Fix hanging stream session (CASSANDRA-10992)
 + * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
 + * Release sstables of failed stream sessions only when outgoing transfers 
are finished (CASSANDRA-11345)
 + * Wait for tracing events before returning response and query at same 
consistency level client side (CASSANDRA-11465)
 + * cqlsh copyutil should get host metadata by connected address 
(CASSANDRA-11979)
 + * Fixed cqlshlib.test.remove_test_db (CASSANDRA-12214)
 +Merged from 2.1:
 + * cannot use cql since upgrading python to 2.7.11+ (CASSANDRA-11850)
 + * Allow STCS-in-L0 compactions to reduce scope with LCS (CASSANDRA-12040)
 +
 +
 +3.8
 + * RTE from new CDC column breaks in flight queries (CASSANDRA-12236)
 + * Fix hdr logging for single operation workloads (CASSANDRA-12145)
 + * Fix SASI PREFIX search in CONTAINS mode with partial terms 
(CASSANDRA-12073)
 + * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 + * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
 + * Upgrade netty to 4.0.39 (CASSANDRA-12032, CASSANDRA-12034)
 + * Improve details in compaction log message (CASSANDRA-12080)
 + * Allow unset values in CQLSSTableWriter (CASSANDRA-11911)
 + * Chunk cache to request compressor-compatible buffers if pool space is 
exhausted (CASSANDRA-11993)
 + * Remove DatabaseDescriptor dependencies from SequentialWriter 
(CASSANDRA-11579)
 + * Move skip_stop_words filter before stemming (CASSANDRA-12078)
 + * Support seek() in EncryptedFileSegmentInputStream (CASSANDRA-11957)
 + * SSTable tools mishandling LocalPartitioner (CASSANDRA-12002)
 + * When SEPWorker assigned work, set thread name to match pool 
(CASSANDRA-11966)
 + * Add cross-DC latency metrics (CASSANDRA-11596)
 + * Allow terms in selection clause (CASSANDRA-10783)
 + * Add bind variables to trace (CASSANDRA-11719)
 + * Switch counter shards' clock to timestamps (CASSANDRA-9811)
 + * Introduce HdrHistogram and response/service/wait separation to stress tool 
(CASSANDRA-11853)
 + * entry-weighers in QueryProcessor should respect partitionKeyBindIndexes 
field (CASSANDRA-11718)
 + * Support older ant versions (CASSANDRA-11807)
 + * Estimate compressed on disk size when deciding if sstable size limit 
reached (CASSANDRA-11623)
 + * cassandra-stress profiles should support case sensitive schemas 
(CASSANDRA-11546)
 + * Remove DatabaseDescriptor dependency from FileUtils (CASSANDRA-11578)
 + * Faster streaming (CASSANDRA-9766)
 + * Add prepared query parameter to trace for "Execute CQL3 prepared query" 
session (CASSANDRA-11425)
 + * Add repaired percentage metric (CASSANDRA-11503)
 + * Add Change-Data-Capture (CASSANDRA-8844)
 +Merged from 3.0:
   * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
   * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
   * Fix migration of static 

[10/10] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[yukim]

Merge branch 'cassandra-3.9' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89a92797
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89a92797
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89a92797

Branch: refs/heads/trunk
Commit: 89a927978fd8686564789d0170af824974bf258b
Parents: 4878852 8458e4e
Author: Yuki Morishita 
Authored: Tue Aug 9 16:56:35 2016 -0500
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:56:35 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +-
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 +++-
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 14 ++
 .../compression/CompressedInputStreamTest.java  | 52 +---
 11 files changed, 100 insertions(+), 95 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/89a92797/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/89a92797/src/java/org/apache/cassandra/config/Config.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/89a92797/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62ef8617
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62ef8617
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62ef8617

Branch: refs/heads/cassandra-3.0
Commit: 62ef8617cdaa07fa37b1b2121ad5923da64e74a3
Parents: 676b6a8 76e3100
Author: Yuki Morishita 
Authored: Tue Aug 9 16:45:52 2016 -0500
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:45:52 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +-
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 +++-
 .../compress/CompressedStreamReader.java| 11 ++---
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 14 ++
 .../compression/CompressedInputStreamTest.java  | 52 +---
 11 files changed, 100 insertions(+), 96 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/62ef8617/CHANGES.txt
--
diff --cc CHANGES.txt
index 78bd32d,232203e..f613c5f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,6 +1,35 @@@
 -2.2.8
 +3.0.9
 + * Change commitlog and sstables to track dirty and clean intervals 
(CASSANDRA-11828)
 + * NullPointerException during compaction on table with static columns 
(CASSANDRA-12336)
 + * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
 + * Fix upgrade of super columns on thrift (CASSANDRA-12335)
 + * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
 + * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
 + * Exception when computing read-repair for range tombstones (CASSANDRA-12263)
 + * Lost counter writes in compact table and static columns (CASSANDRA-12219)
 + * AssertionError with MVs on updating a row that isn't indexed due to a null 
value (CASSANDRA-12247)
 + * Disable RR and speculative retry with EACH_QUORUM reads (CASSANDRA-11980)
 + * Add option to override compaction space check (CASSANDRA-12180)
 + * Faster startup by only scanning each directory for temporary files once 
(CASSANDRA-12114)
 + * Respond with v1/v2 protocol header when responding to driver that attempts
 +   to connect with too low of a protocol version (CASSANDRA-11464)
 + * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
 + * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 + * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
 + * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
 + * Fix migration of static thrift column names with non-text comparators 
(CASSANDRA-12147)
 + * Fix upgrading sparse tables that are incorrectly marked as dense 
(CASSANDRA-11315)
 + * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)
 + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098)
 + * Avoid missing sstables when getting the canonical sstables 
(CASSANDRA-11996)
 + * Always select the live sstables when getting sstables in bounds 
(CASSANDRA-11944)
 + * Fix column ordering of results with static columns for Thrift requests in
 +   a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of
 +   those static columns in query results (CASSANDRA-12123)
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 +Merged from 2.2:
+  * Fix hanging stream session (CASSANDRA-10992)
 - * Add byteman support for testing (CASSANDRA-12377)
   * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
   * Restore JVM metric export for metric reporters (CASSANDRA-12312)
   * Release sstables of failed stream sessions only when outgoing transfers 
are finished (CASSANDRA-11345)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/62ef8617/src/java/org/apache/cassandra/config/Config.java
--
diff --cc src/java/org/apache/cassandra/config/Config.java
index e6c56cb,60daee6..86f1016
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -171,8 -170,11 +171,12 @@@ public 

[04/10] cassandra git commit: Fix hanging stream session

[yukim]

Fix hanging stream session

by preventing CompressedStreamReader from blocking on IOException.
Also removed retry support from streaming.

Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-10992


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76e3100f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76e3100f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76e3100f

Branch: refs/heads/trunk
Commit: 76e3100ffb106cab3cc665404e293c1026e5e65c
Parents: bc9af92
Author: Paulo Motta 
Authored: Thu Jun 23 11:33:54 2016 -0300
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:31:34 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +--
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 -
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 17 +++
 .../compress/CompressedInputStreamTest.java | 49 +---
 11 files changed, 102 insertions(+), 93 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f734476..232203e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.8
+ * Fix hanging stream session (CASSANDRA-10992)
  * Add byteman support for testing (CASSANDRA-12377)
  * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
  * Restore JVM metric export for metric reporters (CASSANDRA-12312)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ede4560..60daee6 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -171,6 +171,10 @@ public class Config
 public volatile Integer compaction_throughput_mb_per_sec = 16;
 public volatile Integer compaction_large_partition_warning_threshold_mb = 
100;
 
+/**
+ * @deprecated retry support removed on CASSANDRA-10992
+ */
+@Deprecated
 public Integer max_streaming_retries = 3;
 
 public volatile Integer stream_throughput_outbound_megabits_per_sec = 200;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index f1acfc4..6e46725 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -957,11 +957,6 @@ public class DatabaseDescriptor
 return conf.cluster_name;
 }
 
-public static int getMaxStreamingRetries()
-{
-return conf.max_streaming_retries;
-}
-
 public static int getStoragePort()
 {
 return Integer.parseInt(System.getProperty("cassandra.storage_port", 
conf.storage_port.toString()));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/streaming/StreamReader.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReader.java 
b/src/java/org/apache/cassandra/streaming/StreamReader.java
index 8789720..c96ea22 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReader.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReader.java
@@ -45,6 +45,7 @@ import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.BytesReadTracker;
 import org.apache.cassandra.utils.Pair;
 
+import static org.apache.cassandra.utils.Throwables.extractIOExceptionCause;
 
 /**
  * StreamReader reads from stream and writes to SSTable.
@@ -137,11 +138,7 @@ public class StreamReader
 e.addSuppressed(e2);
 }
 }
-drain(dis, in.getBytesRead());
-if (e instanceof IOException)
-throw (IOException) e;
-else
-throw Throwables.propagate(e);
+ 

[03/10] cassandra git commit: Fix hanging stream session

[yukim]

Fix hanging stream session

by preventing CompressedStreamReader from blocking on IOException.
Also removed retry support from streaming.

Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-10992


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76e3100f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76e3100f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76e3100f

Branch: refs/heads/cassandra-3.9
Commit: 76e3100ffb106cab3cc665404e293c1026e5e65c
Parents: bc9af92
Author: Paulo Motta 
Authored: Thu Jun 23 11:33:54 2016 -0300
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:31:34 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +--
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 -
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 17 +++
 .../compress/CompressedInputStreamTest.java | 49 +---
 11 files changed, 102 insertions(+), 93 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f734476..232203e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.8
+ * Fix hanging stream session (CASSANDRA-10992)
  * Add byteman support for testing (CASSANDRA-12377)
  * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
  * Restore JVM metric export for metric reporters (CASSANDRA-12312)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ede4560..60daee6 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -171,6 +171,10 @@ public class Config
 public volatile Integer compaction_throughput_mb_per_sec = 16;
 public volatile Integer compaction_large_partition_warning_threshold_mb = 
100;
 
+/**
+ * @deprecated retry support removed on CASSANDRA-10992
+ */
+@Deprecated
 public Integer max_streaming_retries = 3;
 
 public volatile Integer stream_throughput_outbound_megabits_per_sec = 200;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index f1acfc4..6e46725 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -957,11 +957,6 @@ public class DatabaseDescriptor
 return conf.cluster_name;
 }
 
-public static int getMaxStreamingRetries()
-{
-return conf.max_streaming_retries;
-}
-
 public static int getStoragePort()
 {
 return Integer.parseInt(System.getProperty("cassandra.storage_port", 
conf.storage_port.toString()));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/streaming/StreamReader.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReader.java 
b/src/java/org/apache/cassandra/streaming/StreamReader.java
index 8789720..c96ea22 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReader.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReader.java
@@ -45,6 +45,7 @@ import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.BytesReadTracker;
 import org.apache.cassandra.utils.Pair;
 
+import static org.apache.cassandra.utils.Throwables.extractIOExceptionCause;
 
 /**
  * StreamReader reads from stream and writes to SSTable.
@@ -137,11 +138,7 @@ public class StreamReader
 e.addSuppressed(e2);
 }
 }
-drain(dis, in.getBytesRead());
-if (e instanceof IOException)
-throw (IOException) e;
-else
-throw 

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8458e4e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8458e4e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8458e4e3

Branch: refs/heads/trunk
Commit: 8458e4e30744e8df6fcbe12aec286db047411385
Parents: ee60941 62ef861
Author: Yuki Morishita 
Authored: Tue Aug 9 16:56:29 2016 -0500
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:56:29 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +-
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 +++-
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 14 ++
 .../compression/CompressedInputStreamTest.java  | 52 +---
 11 files changed, 100 insertions(+), 95 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8458e4e3/CHANGES.txt
--
diff --cc CHANGES.txt
index b7bbf72,f613c5f..8fbd774
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,47 -15,6 +22,48 @@@ Merged from 3.0
 to connect with too low of a protocol version (CASSANDRA-11464)
   * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
   * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 +Merged from 2.2:
++ * Fix hanging stream session (CASSANDRA-10992)
 + * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
 + * Release sstables of failed stream sessions only when outgoing transfers 
are finished (CASSANDRA-11345)
 + * Wait for tracing events before returning response and query at same 
consistency level client side (CASSANDRA-11465)
 + * cqlsh copyutil should get host metadata by connected address 
(CASSANDRA-11979)
 + * Fixed cqlshlib.test.remove_test_db (CASSANDRA-12214)
 +Merged from 2.1:
 + * cannot use cql since upgrading python to 2.7.11+ (CASSANDRA-11850)
 + * Allow STCS-in-L0 compactions to reduce scope with LCS (CASSANDRA-12040)
 +
 +
 +3.8
 + * RTE from new CDC column breaks in flight queries (CASSANDRA-12236)
 + * Fix hdr logging for single operation workloads (CASSANDRA-12145)
 + * Fix SASI PREFIX search in CONTAINS mode with partial terms 
(CASSANDRA-12073)
 + * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 + * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
 + * Upgrade netty to 4.0.39 (CASSANDRA-12032, CASSANDRA-12034)
 + * Improve details in compaction log message (CASSANDRA-12080)
 + * Allow unset values in CQLSSTableWriter (CASSANDRA-11911)
 + * Chunk cache to request compressor-compatible buffers if pool space is 
exhausted (CASSANDRA-11993)
 + * Remove DatabaseDescriptor dependencies from SequentialWriter 
(CASSANDRA-11579)
 + * Move skip_stop_words filter before stemming (CASSANDRA-12078)
 + * Support seek() in EncryptedFileSegmentInputStream (CASSANDRA-11957)
 + * SSTable tools mishandling LocalPartitioner (CASSANDRA-12002)
 + * When SEPWorker assigned work, set thread name to match pool 
(CASSANDRA-11966)
 + * Add cross-DC latency metrics (CASSANDRA-11596)
 + * Allow terms in selection clause (CASSANDRA-10783)
 + * Add bind variables to trace (CASSANDRA-11719)
 + * Switch counter shards' clock to timestamps (CASSANDRA-9811)
 + * Introduce HdrHistogram and response/service/wait separation to stress tool 
(CASSANDRA-11853)
 + * entry-weighers in QueryProcessor should respect partitionKeyBindIndexes 
field (CASSANDRA-11718)
 + * Support older ant versions (CASSANDRA-11807)
 + * Estimate compressed on disk size when deciding if sstable size limit 
reached (CASSANDRA-11623)
 + * cassandra-stress profiles should support case sensitive schemas 
(CASSANDRA-11546)
 + * Remove DatabaseDescriptor dependency from FileUtils (CASSANDRA-11578)
 + * Faster streaming (CASSANDRA-9766)
 + * Add prepared query parameter to trace for "Execute CQL3 prepared query" 
session (CASSANDRA-11425)
 + * Add repaired percentage metric (CASSANDRA-11503)
 + * Add Change-Data-Capture (CASSANDRA-8844)
 +Merged from 3.0:
   * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
   * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
   * Fix migration of static thrift 

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62ef8617
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62ef8617
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62ef8617

Branch: refs/heads/cassandra-3.9
Commit: 62ef8617cdaa07fa37b1b2121ad5923da64e74a3
Parents: 676b6a8 76e3100
Author: Yuki Morishita 
Authored: Tue Aug 9 16:45:52 2016 -0500
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:45:52 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +-
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 +++-
 .../compress/CompressedStreamReader.java| 11 ++---
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 14 ++
 .../compression/CompressedInputStreamTest.java  | 52 +---
 11 files changed, 100 insertions(+), 96 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/62ef8617/CHANGES.txt
--
diff --cc CHANGES.txt
index 78bd32d,232203e..f613c5f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,6 +1,35 @@@
 -2.2.8
 +3.0.9
 + * Change commitlog and sstables to track dirty and clean intervals 
(CASSANDRA-11828)
 + * NullPointerException during compaction on table with static columns 
(CASSANDRA-12336)
 + * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
 + * Fix upgrade of super columns on thrift (CASSANDRA-12335)
 + * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
 + * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
 + * Exception when computing read-repair for range tombstones (CASSANDRA-12263)
 + * Lost counter writes in compact table and static columns (CASSANDRA-12219)
 + * AssertionError with MVs on updating a row that isn't indexed due to a null 
value (CASSANDRA-12247)
 + * Disable RR and speculative retry with EACH_QUORUM reads (CASSANDRA-11980)
 + * Add option to override compaction space check (CASSANDRA-12180)
 + * Faster startup by only scanning each directory for temporary files once 
(CASSANDRA-12114)
 + * Respond with v1/v2 protocol header when responding to driver that attempts
 +   to connect with too low of a protocol version (CASSANDRA-11464)
 + * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
 + * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 + * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
 + * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
 + * Fix migration of static thrift column names with non-text comparators 
(CASSANDRA-12147)
 + * Fix upgrading sparse tables that are incorrectly marked as dense 
(CASSANDRA-11315)
 + * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)
 + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098)
 + * Avoid missing sstables when getting the canonical sstables 
(CASSANDRA-11996)
 + * Always select the live sstables when getting sstables in bounds 
(CASSANDRA-11944)
 + * Fix column ordering of results with static columns for Thrift requests in
 +   a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of
 +   those static columns in query results (CASSANDRA-12123)
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 +Merged from 2.2:
+  * Fix hanging stream session (CASSANDRA-10992)
 - * Add byteman support for testing (CASSANDRA-12377)
   * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
   * Restore JVM metric export for metric reporters (CASSANDRA-12312)
   * Release sstables of failed stream sessions only when outgoing transfers 
are finished (CASSANDRA-11345)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/62ef8617/src/java/org/apache/cassandra/config/Config.java
--
diff --cc src/java/org/apache/cassandra/config/Config.java
index e6c56cb,60daee6..86f1016
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -171,8 -170,11 +171,12 @@@ public 

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62ef8617
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62ef8617
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62ef8617

Branch: refs/heads/trunk
Commit: 62ef8617cdaa07fa37b1b2121ad5923da64e74a3
Parents: 676b6a8 76e3100
Author: Yuki Morishita 
Authored: Tue Aug 9 16:45:52 2016 -0500
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:45:52 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +-
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 +++-
 .../compress/CompressedStreamReader.java| 11 ++---
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 14 ++
 .../compression/CompressedInputStreamTest.java  | 52 +---
 11 files changed, 100 insertions(+), 96 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/62ef8617/CHANGES.txt
--
diff --cc CHANGES.txt
index 78bd32d,232203e..f613c5f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,6 +1,35 @@@
 -2.2.8
 +3.0.9
 + * Change commitlog and sstables to track dirty and clean intervals 
(CASSANDRA-11828)
 + * NullPointerException during compaction on table with static columns 
(CASSANDRA-12336)
 + * Fixed ConcurrentModificationException when reading metrics in 
GraphiteReporter (CASSANDRA-11823)
 + * Fix upgrade of super columns on thrift (CASSANDRA-12335)
 + * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and 
increased corruption size (CASSANDRA-12359)
 + * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness 
(CASSANDRA-12277)
 + * Exception when computing read-repair for range tombstones (CASSANDRA-12263)
 + * Lost counter writes in compact table and static columns (CASSANDRA-12219)
 + * AssertionError with MVs on updating a row that isn't indexed due to a null 
value (CASSANDRA-12247)
 + * Disable RR and speculative retry with EACH_QUORUM reads (CASSANDRA-11980)
 + * Add option to override compaction space check (CASSANDRA-12180)
 + * Faster startup by only scanning each directory for temporary files once 
(CASSANDRA-12114)
 + * Respond with v1/v2 protocol header when responding to driver that attempts
 +   to connect with too low of a protocol version (CASSANDRA-11464)
 + * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
 + * Fix problem with undeleteable rows on upgrade to new sstable format 
(CASSANDRA-12144)
 + * Fix paging logic for deleted partitions with static columns 
(CASSANDRA-12107)
 + * Wait until the message is being send to decide which serializer must be 
used (CASSANDRA-11393)
 + * Fix migration of static thrift column names with non-text comparators 
(CASSANDRA-12147)
 + * Fix upgrading sparse tables that are incorrectly marked as dense 
(CASSANDRA-11315)
 + * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)
 + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098)
 + * Avoid missing sstables when getting the canonical sstables 
(CASSANDRA-11996)
 + * Always select the live sstables when getting sstables in bounds 
(CASSANDRA-11944)
 + * Fix column ordering of results with static columns for Thrift requests in
 +   a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of
 +   those static columns in query results (CASSANDRA-12123)
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 +Merged from 2.2:
+  * Fix hanging stream session (CASSANDRA-10992)
 - * Add byteman support for testing (CASSANDRA-12377)
   * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
   * Restore JVM metric export for metric reporters (CASSANDRA-12312)
   * Release sstables of failed stream sessions only when outgoing transfers 
are finished (CASSANDRA-11345)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/62ef8617/src/java/org/apache/cassandra/config/Config.java
--
diff --cc src/java/org/apache/cassandra/config/Config.java
index e6c56cb,60daee6..86f1016
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -171,8 -170,11 +171,12 @@@ public class 

[01/10] cassandra git commit: Fix hanging stream session

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 bc9af92e8 -> 76e3100ff
  refs/heads/cassandra-3.0 676b6a891 -> 62ef8617c
  refs/heads/cassandra-3.9 ee609411a -> 8458e4e30
  refs/heads/trunk 4878852fe -> 89a927978


Fix hanging stream session

by preventing CompressedStreamReader from blocking on IOException.
Also removed retry support from streaming.

Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-10992


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76e3100f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76e3100f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76e3100f

Branch: refs/heads/cassandra-2.2
Commit: 76e3100ffb106cab3cc665404e293c1026e5e65c
Parents: bc9af92
Author: Paulo Motta 
Authored: Thu Jun 23 11:33:54 2016 -0300
Committer: Yuki Morishita 
Committed: Tue Aug 9 16:31:34 2016 -0500

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java |  4 ++
 .../cassandra/config/DatabaseDescriptor.java|  5 --
 .../cassandra/streaming/StreamReader.java   | 26 +--
 .../cassandra/streaming/StreamSession.java  | 36 +-
 .../compress/CompressedInputStream.java | 21 -
 .../compress/CompressedStreamReader.java| 10 ++--
 .../streaming/messages/IncomingFileMessage.java | 22 ++---
 .../streaming/messages/RetryMessage.java|  4 ++
 .../org/apache/cassandra/utils/Throwables.java  | 17 +++
 .../compress/CompressedInputStreamTest.java | 49 +---
 11 files changed, 102 insertions(+), 93 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f734476..232203e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.8
+ * Fix hanging stream session (CASSANDRA-10992)
  * Add byteman support for testing (CASSANDRA-12377)
  * Fix INSERT JSON, fromJson() support of smallint, tinyint types 
(CASSANDRA-12371)
  * Restore JVM metric export for metric reporters (CASSANDRA-12312)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ede4560..60daee6 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -171,6 +171,10 @@ public class Config
 public volatile Integer compaction_throughput_mb_per_sec = 16;
 public volatile Integer compaction_large_partition_warning_threshold_mb = 
100;
 
+/**
+ * @deprecated retry support removed on CASSANDRA-10992
+ */
+@Deprecated
 public Integer max_streaming_retries = 3;
 
 public volatile Integer stream_throughput_outbound_megabits_per_sec = 200;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index f1acfc4..6e46725 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -957,11 +957,6 @@ public class DatabaseDescriptor
 return conf.cluster_name;
 }
 
-public static int getMaxStreamingRetries()
-{
-return conf.max_streaming_retries;
-}
-
 public static int getStoragePort()
 {
 return Integer.parseInt(System.getProperty("cassandra.storage_port", 
conf.storage_port.toString()));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/76e3100f/src/java/org/apache/cassandra/streaming/StreamReader.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReader.java 
b/src/java/org/apache/cassandra/streaming/StreamReader.java
index 8789720..c96ea22 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReader.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReader.java
@@ -45,6 +45,7 @@ import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.BytesReadTracker;
 import org.apache.cassandra.utils.Pair;
 
+import static org.apache.cassandra.utils.Throwables.extractIOExceptionCause;
 
 /**
  * StreamReader reads from stream and writes to SSTable.
@@ -137,11 +138,7 @@ public class StreamReader
 

[jira] [Updated] (CASSANDRA-9876) One way targeted repair

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9876:
---
Status: Open  (was: Patch Available)

> One way targeted repair
> ---
>
> Key: CASSANDRA-9876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9876
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 9876-trunk.txt
>
>
> Many applications use C* by writing to one local DC. The other DC is used 
> when the local DC is unavailable. When the local DC becomes available, we 
> want to run a targeted repair b/w one endpoint from each DC to minimize the 
> data transfer over WAN.  In this case, it will be helpful to do a one way 
> repair in which data will only be streamed from other DC to local DC instead 
> of streaming the data both ways. This will further minimize the traffic over 
> WAN. This feature should only be supported if a targeted repair is run 
> involving 2 hosts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9876) One way targeted repair

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414245#comment-15414245
 ] 

Paulo Motta commented on CASSANDRA-9876:


Thanks for the patch! Overall this looks good and nearly ready, just a few 
minor nits to fix:
* Since {{pullRepair}} is coordinator-only, I think it's better to pass it 
directly to {{LocalSyncTask}} instead of passing it via {{RepairJobDesc}} which 
is also a remote object (I saw that you are not serializing it, but I think 
it's cleaner to keep this to the coordinator only to avoid confusion).
* since the command is already called repair, perhaps we should call the option 
{{\-\-pull}} instead of {{\-\-pull-repair}}?
* I don't think it's necessary to make specifying {{\-\-start-token}} and 
{{\-\-end-token}} mandatory, since if that is not specified it will just pull 
repair all common ranges between specified hosts.
* can you just a simple dtest on {{repair_tests/repair_test.py}} verifying this 
works?

> One way targeted repair
> ---
>
> Key: CASSANDRA-9876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9876
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 9876-trunk.txt
>
>
> Many applications use C* by writing to one local DC. The other DC is used 
> when the local DC is unavailable. When the local DC becomes available, we 
> want to run a targeted repair b/w one endpoint from each DC to minimize the 
> data transfer over WAN.  In this case, it will be helpful to do a one way 
> repair in which data will only be streamed from other DC to local DC instead 
> of streaming the data both ways. This will further minimize the traffic over 
> WAN. This feature should only be supported if a targeted repair is run 
> involving 2 hosts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12311) Propagate TombstoneOverwhelmingException to the client

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414227#comment-15414227
 ] 

Tyler Hobbs commented on CASSANDRA-12311:
-

Thanks! The tests are pretty much good to go.  I made some minor tweaks and 
pushed a branch here: 
https://github.com/thobbs/cassandra-dtest/tree/CASSANDRA-12311-tests

Once the python driver changes are available for cassci to use, I'll schedule 
test runs (should be later this week).

> Propagate TombstoneOverwhelmingException to the client
> --
>
> Key: CASSANDRA-12311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12311
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
>  Labels: client-impacting, doc-impacting
> Fix For: 4.x
>
> Attachments: 12311-dtest.txt, 12311-trunk-v2.txt, 12311-trunk-v3.txt, 
> 12311-trunk-v4.txt, 12311-trunk-v5.txt, 12311-trunk.txt
>
>
> Right now if a data node fails to perform a read because it ran into a 
> {{TombstoneOverwhelmingException}}, it only responds back to the coordinator 
> node with a generic failure. Under this scheme, the coordinator won't be able 
> to know exactly why the request failed and subsequently the client only gets 
> a generic {{ReadFailureException}}. It would be useful to inform the client 
> that their read failed because we read too many tombstones. We should have 
> the data nodes reply with a failure type so the coordinator can pass this 
> information to the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12208) Estimated droppable tombstones given by sstablemetadata counts tombstones that aren't actually "droppable"

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-12208:
---
 Reviewer: Yuki Morishita
Fix Version/s: (was: 3.0.9)
   (was: 3.9)
   3.x
   3.0.x
  Component/s: Tools

> Estimated droppable tombstones given by sstablemetadata counts tombstones 
> that aren't actually "droppable"
> --
>
> Key: CASSANDRA-12208
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12208
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Thanh
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> => "Estimated droppable tombstones" given by *sstablemetadata* counts 
> tombstones that aren't actually "droppable"
> To be clear, the "Estimated droppable tombstones" calculation counts 
> tombstones that have not yet passed gc_grace_seconds as droppable tombstones, 
> which is unexpected, since such tombstones aren't droppable.
> To observe the problem:
> Create a table using the default gc_grace_seconds (default gc_grace_seconds 
> is 86400 is 1 day).
> Populate the table with a couple of records.
> Do a delete.
> Do a "nodetool flush" to flush the memtable to disk.
> Do an "sstablemetadata " to get the metadata of the sstable you just 
> created by doing the flush, and observe that the Estimated droppable 
> tombstones is greater than 0.0 (actual value depends on the total number 
> inserts/updates/deletes that you did before triggered the flush)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12283) CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414157#comment-15414157
 ] 

Benjamin Lerer commented on CASSANDRA-12283:


In my opinion the condition should be either: {{current millis - start millis < 
timeout millis}} or {{current millis < start millis + timeout millis}} but the 
current one does not make any sense to me (it was already changed by [~blambov] 
in trunk).

The test seems to timeout randomly. I could not reproduce it on my Windows 
machine so I was looking at what could introduce some random delays. According 
to [this|http://www.javamex.com/tutorials/threads/yield.shtml] 
{{Thread.yield()}} seems to be able to result into some random delay. By 
consequence I though that it might be good to remove it anyway.

> CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky
> 
>
> Key: CASSANDRA-12283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12283
> Project: Cassandra
>  Issue Type: Test
>Reporter: Joshua McKenzie
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: unittest
>
> Failed 3 of the last 38 runs.
> [Failure|http://cassci.datastax.com/job/cassandra-3.9_testall/lastCompletedBuild/testReport/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerTest/testCompressedCommitLogBackpressure/]
> Details:
> Error Message
> Timeout occurred. Please note the time in the report does not reflect the 
> time until the timeout.
> Stacktrace
> junit.framework.AssertionFailedError: Timeout occurred. Please note the time 
> in the report does not reflect the time until the timeout.
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414153#comment-15414153
 ] 

Tyler Hobbs commented on CASSANDRA-12249:
-

I discussed this problem some with Benjamin.  Here's a summary of what's 
happening:

The 3.x used to never send a {{PAGED_RANGE}} verb message.  So, for 
backwards-compatibility handling, we always assumed that {{PAGED_RANGE}} 
messages came from 2.x nodes.  However, CASSANDRA-11393 caused 3.x to start 
using {{PAGED_RANGE}} again.  So, when the 3.0.8 node gets a {{PAGED_RANGE}} 
message from the 3.8 node, it thinks it's from a 2.x node.

The ideal fix is to make 3.x stop using {{PAGED_RANGE}} again.  Without having 
looked into the code too deeply around this, it seems like we might have to 
pick the verb for the message at a later point in order to make this work.

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11363) High Blocked NTR When Connecting

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11363:

Assignee: T Jake Luciani  (was: Paulo Motta)

> High Blocked NTR When Connecting
> 
>
> Key: CASSANDRA-11363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11363
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Russell Bradberry
>Assignee: T Jake Luciani
> Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack, 
> max_queued_ntr_property.txt, thread-queue-2.1.txt
>
>
> When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the 
> machine load increases to very high levels (> 120 on an 8 core machine) and 
> native transport requests get blocked in tpstats.
> I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8.
> The issue does not seem to affect the nodes running 2.1.9.
> The issue seems to coincide with the number of connections OR the number of 
> total requests being processed at a given time (as the latter increases with 
> the former in our system)
> Currently there is between 600 and 800 client connections on each machine and 
> each machine is handling roughly 2000-3000 client requests per second.
> Disabling the binary protocol fixes the issue for this node but isn't a 
> viable option cluster-wide.
> Here is the output from tpstats:
> {code}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> MutationStage 0 88387821 0
>  0
> ReadStage 0 0 355860 0
>  0
> RequestResponseStage  0 72532457 0
>  0
> ReadRepairStage   0 0150 0
>  0
> CounterMutationStage 32   104 897560 0
>  0
> MiscStage 0 0  0 0
>  0
> HintedHandoff 0 0 65 0
>  0
> GossipStage   0 0   2338 0
>  0
> CacheCleanupExecutor  0 0  0 0
>  0
> InternalResponseStage 0 0  0 0
>  0
> CommitLogArchiver 0 0  0 0
>  0
> CompactionExecutor2   190474 0
>  0
> ValidationExecutor0 0  0 0
>  0
> MigrationStage0 0 10 0
>  0
> AntiEntropyStage  0 0  0 0
>  0
> PendingRangeCalculator0 0310 0
>  0
> Sampler   0 0  0 0
>  0
> MemtableFlushWriter   110 94 0
>  0
> MemtablePostFlush 134257 0
>  0
> MemtableReclaimMemory 0 0 94 0
>  0
> Native-Transport-Requests   128   156 38795716
> 278451
> Message type   Dropped
> READ 0
> RANGE_SLICE  0
> _TRACE   0
> MUTATION 0
> COUNTER_MUTATION 0
> BINARY   0
> REQUEST_RESPONSE 0
> PAGED_RANGE  0
> READ_REPAIR  0
> {code}
> Attached is the jstack output for both CMS and G1GC.
> Flight recordings are here:
> https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr
> https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr
> It is interesting to note that while the flight recording was taking place, 
> the load on the machine went back to healthy, and when the flight recording 
> finished the load went back to > 100.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11363) High Blocked NTR When Connecting

[Nate McCall (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414132#comment-15414132
 ] 

Nate McCall edited comment on CASSANDRA-11363 at 8/9/16 7:55 PM:
-

Sorry, but I am re-opening this. Given we have a potential explanation and 
workaround available and that I still see blocked NTR (to a varying degree) on 
*every* 2.x+ cluster I have come in contact with recently. 


was (Author: zznate):
Sorry, but I am re-opening this. With have a potential explanation with 
workaround and I still see it with varying degrees on *every* 2.x+ cluster we 
have come in contact with recently. 

> High Blocked NTR When Connecting
> 
>
> Key: CASSANDRA-11363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11363
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Russell Bradberry
>Assignee: Paulo Motta
> Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack, 
> max_queued_ntr_property.txt, thread-queue-2.1.txt
>
>
> When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the 
> machine load increases to very high levels (> 120 on an 8 core machine) and 
> native transport requests get blocked in tpstats.
> I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8.
> The issue does not seem to affect the nodes running 2.1.9.
> The issue seems to coincide with the number of connections OR the number of 
> total requests being processed at a given time (as the latter increases with 
> the former in our system)
> Currently there is between 600 and 800 client connections on each machine and 
> each machine is handling roughly 2000-3000 client requests per second.
> Disabling the binary protocol fixes the issue for this node but isn't a 
> viable option cluster-wide.
> Here is the output from tpstats:
> {code}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> MutationStage 0 88387821 0
>  0
> ReadStage 0 0 355860 0
>  0
> RequestResponseStage  0 72532457 0
>  0
> ReadRepairStage   0 0150 0
>  0
> CounterMutationStage 32   104 897560 0
>  0
> MiscStage 0 0  0 0
>  0
> HintedHandoff 0 0 65 0
>  0
> GossipStage   0 0   2338 0
>  0
> CacheCleanupExecutor  0 0  0 0
>  0
> InternalResponseStage 0 0  0 0
>  0
> CommitLogArchiver 0 0  0 0
>  0
> CompactionExecutor2   190474 0
>  0
> ValidationExecutor0 0  0 0
>  0
> MigrationStage0 0 10 0
>  0
> AntiEntropyStage  0 0  0 0
>  0
> PendingRangeCalculator0 0310 0
>  0
> Sampler   0 0  0 0
>  0
> MemtableFlushWriter   110 94 0
>  0
> MemtablePostFlush 134257 0
>  0
> MemtableReclaimMemory 0 0 94 0
>  0
> Native-Transport-Requests   128   156 38795716
> 278451
> Message type   Dropped
> READ 0
> RANGE_SLICE  0
> _TRACE   0
> MUTATION 0
> COUNTER_MUTATION 0
> BINARY   0
> REQUEST_RESPONSE 0
> PAGED_RANGE  0
> READ_REPAIR  0
> {code}
> Attached is the jstack output for both CMS and G1GC.
> Flight recordings are here:
> https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr
> https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr
> It is interesting to note that while the flight recording was taking place, 
> the load on the machine went back to healthy, and when the flight recording 
> finished the load went back to > 100.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11363) High Blocked NTR When Connecting

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414134#comment-15414134
 ] 

T Jake Luciani commented on CASSANDRA-11363:


Great, looks like this is the issue.  I guess the question is what is a 
reasonable default for this? Should we set it high? what's too high?  

Any suggestions [~benedict]?

> High Blocked NTR When Connecting
> 
>
> Key: CASSANDRA-11363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11363
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Russell Bradberry
>Assignee: Paulo Motta
> Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack, 
> max_queued_ntr_property.txt, thread-queue-2.1.txt
>
>
> When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the 
> machine load increases to very high levels (> 120 on an 8 core machine) and 
> native transport requests get blocked in tpstats.
> I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8.
> The issue does not seem to affect the nodes running 2.1.9.
> The issue seems to coincide with the number of connections OR the number of 
> total requests being processed at a given time (as the latter increases with 
> the former in our system)
> Currently there is between 600 and 800 client connections on each machine and 
> each machine is handling roughly 2000-3000 client requests per second.
> Disabling the binary protocol fixes the issue for this node but isn't a 
> viable option cluster-wide.
> Here is the output from tpstats:
> {code}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> MutationStage 0 88387821 0
>  0
> ReadStage 0 0 355860 0
>  0
> RequestResponseStage  0 72532457 0
>  0
> ReadRepairStage   0 0150 0
>  0
> CounterMutationStage 32   104 897560 0
>  0
> MiscStage 0 0  0 0
>  0
> HintedHandoff 0 0 65 0
>  0
> GossipStage   0 0   2338 0
>  0
> CacheCleanupExecutor  0 0  0 0
>  0
> InternalResponseStage 0 0  0 0
>  0
> CommitLogArchiver 0 0  0 0
>  0
> CompactionExecutor2   190474 0
>  0
> ValidationExecutor0 0  0 0
>  0
> MigrationStage0 0 10 0
>  0
> AntiEntropyStage  0 0  0 0
>  0
> PendingRangeCalculator0 0310 0
>  0
> Sampler   0 0  0 0
>  0
> MemtableFlushWriter   110 94 0
>  0
> MemtablePostFlush 134257 0
>  0
> MemtableReclaimMemory 0 0 94 0
>  0
> Native-Transport-Requests   128   156 38795716
> 278451
> Message type   Dropped
> READ 0
> RANGE_SLICE  0
> _TRACE   0
> MUTATION 0
> COUNTER_MUTATION 0
> BINARY   0
> REQUEST_RESPONSE 0
> PAGED_RANGE  0
> READ_REPAIR  0
> {code}
> Attached is the jstack output for both CMS and G1GC.
> Flight recordings are here:
> https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr
> https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr
> It is interesting to note that while the flight recording was taking place, 
> the load on the machine went back to healthy, and when the flight recording 
> finished the load went back to > 100.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-11363) High Blocked NTR When Connecting

[Nate McCall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nate McCall reopened CASSANDRA-11363:
-
Reproduced In: 3.0.3, 2.1.13, 2.1.12  (was: 2.1.12, 2.1.13, 3.0.3)

Sorry, but I am re-opening this. With have a potential explanation with 
workaround and I still see it with varying degrees on *every* 2.x+ cluster we 
have come in contact with recently. 

> High Blocked NTR When Connecting
> 
>
> Key: CASSANDRA-11363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11363
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Russell Bradberry
>Assignee: Paulo Motta
> Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack, 
> max_queued_ntr_property.txt, thread-queue-2.1.txt
>
>
> When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the 
> machine load increases to very high levels (> 120 on an 8 core machine) and 
> native transport requests get blocked in tpstats.
> I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8.
> The issue does not seem to affect the nodes running 2.1.9.
> The issue seems to coincide with the number of connections OR the number of 
> total requests being processed at a given time (as the latter increases with 
> the former in our system)
> Currently there is between 600 and 800 client connections on each machine and 
> each machine is handling roughly 2000-3000 client requests per second.
> Disabling the binary protocol fixes the issue for this node but isn't a 
> viable option cluster-wide.
> Here is the output from tpstats:
> {code}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> MutationStage 0 88387821 0
>  0
> ReadStage 0 0 355860 0
>  0
> RequestResponseStage  0 72532457 0
>  0
> ReadRepairStage   0 0150 0
>  0
> CounterMutationStage 32   104 897560 0
>  0
> MiscStage 0 0  0 0
>  0
> HintedHandoff 0 0 65 0
>  0
> GossipStage   0 0   2338 0
>  0
> CacheCleanupExecutor  0 0  0 0
>  0
> InternalResponseStage 0 0  0 0
>  0
> CommitLogArchiver 0 0  0 0
>  0
> CompactionExecutor2   190474 0
>  0
> ValidationExecutor0 0  0 0
>  0
> MigrationStage0 0 10 0
>  0
> AntiEntropyStage  0 0  0 0
>  0
> PendingRangeCalculator0 0310 0
>  0
> Sampler   0 0  0 0
>  0
> MemtableFlushWriter   110 94 0
>  0
> MemtablePostFlush 134257 0
>  0
> MemtableReclaimMemory 0 0 94 0
>  0
> Native-Transport-Requests   128   156 38795716
> 278451
> Message type   Dropped
> READ 0
> RANGE_SLICE  0
> _TRACE   0
> MUTATION 0
> COUNTER_MUTATION 0
> BINARY   0
> REQUEST_RESPONSE 0
> PAGED_RANGE  0
> READ_REPAIR  0
> {code}
> Attached is the jstack output for both CMS and G1GC.
> Flight recordings are here:
> https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr
> https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr
> It is interesting to note that while the flight recording was taking place, 
> the load on the machine went back to healthy, and when the flight recording 
> finished the load went back to > 100.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11363) High Blocked NTR When Connecting

[Nate McCall (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414131#comment-15414131
 ] 

Nate McCall commented on CASSANDRA-11363:
-

[~tjake] We are preparing this patch for a deploy to two different cluster: 
2.2.6 and 2.1.16. Both clusters can exhibit high-burst workloads (which is why 
I think you can't reproduce this with the stress tool, and I agree with your 
suspicions on the cause). We'll let you know how it goes. 

> High Blocked NTR When Connecting
> 
>
> Key: CASSANDRA-11363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11363
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Russell Bradberry
>Assignee: Paulo Motta
> Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack, 
> max_queued_ntr_property.txt, thread-queue-2.1.txt
>
>
> When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the 
> machine load increases to very high levels (> 120 on an 8 core machine) and 
> native transport requests get blocked in tpstats.
> I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8.
> The issue does not seem to affect the nodes running 2.1.9.
> The issue seems to coincide with the number of connections OR the number of 
> total requests being processed at a given time (as the latter increases with 
> the former in our system)
> Currently there is between 600 and 800 client connections on each machine and 
> each machine is handling roughly 2000-3000 client requests per second.
> Disabling the binary protocol fixes the issue for this node but isn't a 
> viable option cluster-wide.
> Here is the output from tpstats:
> {code}
> Pool NameActive   Pending  Completed   Blocked  All 
> time blocked
> MutationStage 0 88387821 0
>  0
> ReadStage 0 0 355860 0
>  0
> RequestResponseStage  0 72532457 0
>  0
> ReadRepairStage   0 0150 0
>  0
> CounterMutationStage 32   104 897560 0
>  0
> MiscStage 0 0  0 0
>  0
> HintedHandoff 0 0 65 0
>  0
> GossipStage   0 0   2338 0
>  0
> CacheCleanupExecutor  0 0  0 0
>  0
> InternalResponseStage 0 0  0 0
>  0
> CommitLogArchiver 0 0  0 0
>  0
> CompactionExecutor2   190474 0
>  0
> ValidationExecutor0 0  0 0
>  0
> MigrationStage0 0 10 0
>  0
> AntiEntropyStage  0 0  0 0
>  0
> PendingRangeCalculator0 0310 0
>  0
> Sampler   0 0  0 0
>  0
> MemtableFlushWriter   110 94 0
>  0
> MemtablePostFlush 134257 0
>  0
> MemtableReclaimMemory 0 0 94 0
>  0
> Native-Transport-Requests   128   156 38795716
> 278451
> Message type   Dropped
> READ 0
> RANGE_SLICE  0
> _TRACE   0
> MUTATION 0
> COUNTER_MUTATION 0
> BINARY   0
> REQUEST_RESPONSE 0
> PAGED_RANGE  0
> READ_REPAIR  0
> {code}
> Attached is the jstack output for both CMS and G1GC.
> Flight recordings are here:
> https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr
> https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr
> It is interesting to note that while the flight recording was taking place, 
> the load on the machine went back to healthy, and when the flight recording 
> finished the load went back to > 100.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414098#comment-15414098
 ] 

Benjamin Lerer commented on CASSANDRA-12249:


The patch for CASSANDRA-11393 changes the way the communication is done between 
3.x servers. This patch was supposed to go into 3.0.9 and 3.9, allowing the 2 
versions to communicate together, but it seems that it went to 3.0.8 tentative. 
I am not sure how this happened as I thought I commited it into 3.9 before 
going in holidays.
I am also not sure how to fix it. Ideally, we should revert the patch in 3.8. 

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12407) dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414055#comment-15414055
 ] 

Tyler Hobbs commented on CASSANDRA-12407:
-

+1 to skipping on 2.1

> dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test
> 
>
> Key: CASSANDRA-12407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12407
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/381/testReport/cql_tracing_test/TestCqlTracing/tracing_simple_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 102, in 
> tracing_simple_test
> self.trace(session)
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 74, in 
> trace
> self.assertIn('/127.0.0.1', out)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> '\'/127.0.0.1\' not found in "Consistency level set to ALL.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11701) [windows] dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414042#comment-15414042
 ] 

Tyler Hobbs commented on CASSANDRA-11701:
-

bq. I'm not sure if we want this fix in 2.1 or not, it is not a critical bug 
but it is a regression compared to the old cqlsh COPY functionality. It is a 
rare failure but it can occur if the main thread of a child process needs to 
send an error when the receiving thread is already sending results.

Given that the error is rare, obvious, and easy to recover from (simply re-run 
the COPY command), I suggest limiting the patch to 2.2+.

> [windows] dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_skip_and_max_rows
> -
>
> Key: CASSANDRA-11701
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11701
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Stefania
>  Labels: dtest, windows
>
> looks to be an assertion problem, so could be test or cassandra related:
> e.g.:
> {noformat}
> 1 != 331
> {noformat}
> http://cassci.datastax.com/job/trunk_dtest_win32/404/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_with_skip_and_max_rows
> Failed on CassCI build trunk_dtest_win32 #404



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12283) CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12283:

Reviewer: Joshua McKenzie

> CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky
> 
>
> Key: CASSANDRA-12283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12283
> Project: Cassandra
>  Issue Type: Test
>Reporter: Joshua McKenzie
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: unittest
>
> Failed 3 of the last 38 runs.
> [Failure|http://cassci.datastax.com/job/cassandra-3.9_testall/lastCompletedBuild/testReport/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerTest/testCompressedCommitLogBackpressure/]
> Details:
> Error Message
> Timeout occurred. Please note the time in the report does not reflect the 
> time until the timeout.
> Stacktrace
> junit.framework.AssertionFailedError: Timeout occurred. Please note the time 
> in the report does not reflect the time until the timeout.
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12283) CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414028#comment-15414028
 ] 

Joshua McKenzie commented on CASSANDRA-12283:
-

Not sure I follow the claim that the loop will not run for the correct amount 
of time. Reference:
{code:title=loop}
public static void spinAssertEquals(Object expected, Supplier s, 
int timeoutInSeconds)
{
long now = System.currentTimeMillis();
while (System.currentTimeMillis() - now < now + (1000 * 
timeoutInSeconds))
{
if (s.get().equals(expected))
break;
Thread.yield();
{code}
As {{now}} is set once and then referenced on subsequent iterations, that loop 
should continue as long as the {{current millis - start millis < start millis + 
timeout millis}}.

As for the Thread.yield part of things - I'm mostly neutral on it. It's not a 
correctness issue and, while theoretically the test could eat cycles designated 
for other tests or things, the entire premise is that it should yield in the 
scheduler during its quanta. As we only reference that method in unit tests, it 
doesn't seem like that would be the issue.

> CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure is flaky
> 
>
> Key: CASSANDRA-12283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12283
> Project: Cassandra
>  Issue Type: Test
>Reporter: Joshua McKenzie
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: unittest
>
> Failed 3 of the last 38 runs.
> [Failure|http://cassci.datastax.com/job/cassandra-3.9_testall/lastCompletedBuild/testReport/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerTest/testCompressedCommitLogBackpressure/]
> Details:
> Error Message
> Timeout occurred. Please note the time in the report does not reflect the 
> time until the timeout.
> Stacktrace
> junit.framework.AssertionFailedError: Timeout occurred. Please note the time 
> in the report does not reflect the time until the timeout.
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12421) Add the option to only gossip manual severity, not severity from IOWait.

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-12421:
--
Attachment: 0001-Add-the-option-to-ignore-load-severity.patch

> Add the option to only gossip manual severity, not severity from IOWait.
> 
>
> Key: CASSANDRA-12421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12421
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
> Fix For: 2.1.14
>
> Attachments: 0001-Add-the-option-to-ignore-load-severity.patch
>
>
> Similar to CASSANDRA-11737, but I'd like to still respect the manual 
> severity, and ignore the severity calculated from IOWait/Compaction, since in 
> general disk throughput is not a problem for flash card.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12421) Add the option to only gossip manual severity, not severity from IOWait.

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-12421:
--
 Assignee: Dikang Gu
Fix Version/s: 2.1.14
   Status: Patch Available  (was: Open)

> Add the option to only gossip manual severity, not severity from IOWait.
> 
>
> Key: CASSANDRA-12421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12421
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 2.1.14
>
> Attachments: 0001-Add-the-option-to-ignore-load-severity.patch
>
>
> Similar to CASSANDRA-11737, but I'd like to still respect the manual 
> severity, and ignore the severity calculated from IOWait/Compaction, since in 
> general disk throughput is not a problem for flash card.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12421) Add the option to only gossip manual severity, not severity from IOWait.

[Dikang Gu (JIRA)]

Dikang Gu created CASSANDRA-12421:
-

 Summary: Add the option to only gossip manual severity, not 
severity from IOWait.
 Key: CASSANDRA-12421
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12421
 Project: Cassandra
  Issue Type: Improvement
Reporter: Dikang Gu


Similar to CASSANDRA-11737, but I'd like to still respect the manual severity, 
and ignore the severity calculated from IOWait/Compaction, since in general 
disk throughput is not a problem for flash card.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12256) Properly respect the request timeouts

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413908#comment-15413908
 ] 

sankalp kohli commented on CASSANDRA-12256:
---

cc [~thobbs] Can you please review this? 

> Properly respect the request timeouts
> -
>
> Key: CASSANDRA-12256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12256
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Geoffrey Yu
> Fix For: 3.x
>
> Attachments: 12256-trunk.txt
>
>
> We have a number of {{request_timeout_*}} option, that probably every user 
> expect to be an upper bound on how long the coordinator will wait before 
> timeouting a request, but it's actually not always the case, especially for 
> read requests.
> I believe we don't respect those timeout properly in at least the following 
> cases:
> * On a digest mismatch: in that case, we reset the timeout for the data 
> query, which means the overall query might take up to twice the configured 
> timeout before timeouting.
> * On a range query: the timeout is reset for every sub-range that is queried. 
> With many nodes and vnodes, a range query could span tons of sub-range and so 
> a range query could take pretty much arbitrary long before actually 
> timeouting for the user.
> * On short reads: we also reset the timeout for every short reads "retries".
> It's also worth noting that even outside those, the timeouts don't take most 
> of the processing done by the coordinator (query parsing and CQL handling for 
> instance) into account.
> Now, in all fairness, the reason this is this way is that the timeout 
> currently are *not* timeout for the full user request, but rather how long a 
> coordinator should wait on any given replica for any given internal query 
> before giving up. *However*, I'm pretty sure this is not what user 
> intuitively expect and want, *especially* in the context of CASSANDRA-2848 
> where the goal is explicitely to have an upper bound on the query from the 
> user point of view.
> So I'm suggesting we change how those timeouts are handled to really be 
> timeouts on the whole user query.
> And by that I basically just mean that we'd mark the start of each query as 
> soon as possible in the processing, and use that starting time as base in 
> {{ReadCallback.await}} and {{AbstractWriteResponseHandler.get()}}. It won't 
> be perfect in the sense that we'll still only possibly timeout during 
> "blocking" operations, so typically if parsing a query takes more than your 
> timeout, you still won't timeout until that query is sent, but I think that's 
> probably fine in practice because 1) if you timeouts are small enough that 
> this matter, you're probably doing it wrong and 2) we can totally improve on 
> that later if needs be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11870) Consider allocating direct buffers bypassing ByteBuffer.allocateDirect

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-11870:
-
Status: Patch Available  (was: Open)

> Consider allocating direct buffers bypassing ByteBuffer.allocateDirect
> --
>
> Key: CASSANDRA-11870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11870
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> As outlined in CASSANDRA-11818, {{ByteBuffer.allocateDirect}} uses 
> {{Bits.reserveMemory}}, which is there to respect the JVM setting 
> {{-XX:MaxDirectMemorySize=...}}.
> {{Bits.reserveMemory}} first tries an "optimistic" {{tryReserveMemory}} and 
> exits immediately on success. However, if that somehow doesn't succeed, it 
> triggers a {{System.gc()}}, which is bad IMO (however, kind of how direct 
> buffers work in Java). After that GC it sleeps and tries to reserve the 
> memory up to 9 times - up to 511 ms - and then throws 
> {{OutOfMemoryError("Direct buffer memory")}}.
> This is unnecessary for us since we always immediately "free" direct buffers 
> as soon as we no longer need them.
> Proposal: Manage direct-memory reservations in our own code and skip 
> {{Bits.reserveMemory}} that way.
> (However, Netty direct buffers are not under our control.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12256) Properly respect the request timeouts

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-12256:

Attachment: 12256-trunk.txt

I've attached a first pass at this ticket. The majority of the changes are to 
pass down the query start timestamp all the way to the {{ReadCallback}} and 
{{AbstractWriteResponseHandler}}. The timestamp is recorded when the 
{{QueryState}} is created for a particular query.

> Properly respect the request timeouts
> -
>
> Key: CASSANDRA-12256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12256
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Geoffrey Yu
> Fix For: 3.x
>
> Attachments: 12256-trunk.txt
>
>
> We have a number of {{request_timeout_*}} option, that probably every user 
> expect to be an upper bound on how long the coordinator will wait before 
> timeouting a request, but it's actually not always the case, especially for 
> read requests.
> I believe we don't respect those timeout properly in at least the following 
> cases:
> * On a digest mismatch: in that case, we reset the timeout for the data 
> query, which means the overall query might take up to twice the configured 
> timeout before timeouting.
> * On a range query: the timeout is reset for every sub-range that is queried. 
> With many nodes and vnodes, a range query could span tons of sub-range and so 
> a range query could take pretty much arbitrary long before actually 
> timeouting for the user.
> * On short reads: we also reset the timeout for every short reads "retries".
> It's also worth noting that even outside those, the timeouts don't take most 
> of the processing done by the coordinator (query parsing and CQL handling for 
> instance) into account.
> Now, in all fairness, the reason this is this way is that the timeout 
> currently are *not* timeout for the full user request, but rather how long a 
> coordinator should wait on any given replica for any given internal query 
> before giving up. *However*, I'm pretty sure this is not what user 
> intuitively expect and want, *especially* in the context of CASSANDRA-2848 
> where the goal is explicitely to have an upper bound on the query from the 
> user point of view.
> So I'm suggesting we change how those timeouts are handled to really be 
> timeouts on the whole user query.
> And by that I basically just mean that we'd mark the start of each query as 
> soon as possible in the processing, and use that starting time as base in 
> {{ReadCallback.await}} and {{AbstractWriteResponseHandler.get()}}. It won't 
> be perfect in the sense that we'll still only possibly timeout during 
> "blocking" operations, so typically if parsing a query takes more than your 
> timeout, you still won't timeout until that query is sent, but I think that's 
> probably fine in practice because 1) if you timeouts are small enough that 
> this matter, you're probably doing it wrong and 2) we can totally improve on 
> that later if needs be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12256) Properly respect the request timeouts

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-12256:

Fix Version/s: 3.x
   Status: Patch Available  (was: Open)

> Properly respect the request timeouts
> -
>
> Key: CASSANDRA-12256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12256
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Geoffrey Yu
> Fix For: 3.x
>
> Attachments: 12256-trunk.txt
>
>
> We have a number of {{request_timeout_*}} option, that probably every user 
> expect to be an upper bound on how long the coordinator will wait before 
> timeouting a request, but it's actually not always the case, especially for 
> read requests.
> I believe we don't respect those timeout properly in at least the following 
> cases:
> * On a digest mismatch: in that case, we reset the timeout for the data 
> query, which means the overall query might take up to twice the configured 
> timeout before timeouting.
> * On a range query: the timeout is reset for every sub-range that is queried. 
> With many nodes and vnodes, a range query could span tons of sub-range and so 
> a range query could take pretty much arbitrary long before actually 
> timeouting for the user.
> * On short reads: we also reset the timeout for every short reads "retries".
> It's also worth noting that even outside those, the timeouts don't take most 
> of the processing done by the coordinator (query parsing and CQL handling for 
> instance) into account.
> Now, in all fairness, the reason this is this way is that the timeout 
> currently are *not* timeout for the full user request, but rather how long a 
> coordinator should wait on any given replica for any given internal query 
> before giving up. *However*, I'm pretty sure this is not what user 
> intuitively expect and want, *especially* in the context of CASSANDRA-2848 
> where the goal is explicitely to have an upper bound on the query from the 
> user point of view.
> So I'm suggesting we change how those timeouts are handled to really be 
> timeouts on the whole user query.
> And by that I basically just mean that we'd mark the start of each query as 
> soon as possible in the processing, and use that starting time as base in 
> {{ReadCallback.await}} and {{AbstractWriteResponseHandler.get()}}. It won't 
> be perfect in the sense that we'll still only possibly timeout during 
> "blocking" operations, so typically if parsing a query takes more than your 
> timeout, you still won't timeout until that query is sent, but I think that's 
> probably fine in practice because 1) if you timeouts are small enough that 
> this matter, you're probably doing it wrong and 2) we can totally improve on 
> that later if needs be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12249) dtest failure in upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-12249:
---

Assignee: Tyler Hobbs  (was: Benjamin Lerer)

> dtest failure in 
> upgrade_tests.paging_test.TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x.basic_paging_test
> ---
>
> Key: CASSANDRA-12249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12249
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.8_dtest_upgrade/1/testReport/upgrade_tests.paging_test/TestPagingDataNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x/basic_paging_test
> Failed on CassCI build cassandra-3.8_dtest_upgrade #1
> This is on a mixed version cluster, one node is 3.0.8 and the other is 
> 3.8-tentative.
> Stack trace looks like:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-07-20 04:51:02,836 
> CassandraDaemon.java:201 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:1042)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyPagedRangeCommandSerializer.deserialize(ReadCommand.java:964)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
> {code}
> This trace is from the 3.0.8 node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12420) Duplicated Key in IN clause with a small fetch size will run forever

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-12420:
-
Description: 
This can be easily reproduced and fetch size is smaller than the correct number 
of rows.

A table has 2 partition key, 1 clustering key, 1 column.

>Select select = QueryBuilder.select().from("ks", "cf");
>select.where().and(QueryBuilder.eq("a", 1));
>select.where().and(QueryBuilder.in("b", Arrays.asList(1, 1, 1)));
>select.setFetchSize(5);

Now we put a distinct method in client side to eliminate the duplicated key, 
but it's better to fix inside Cassandra.

  was:
This can be easily reproduced and fetch size is smaller than the correct number 
of rows.
Select select = QueryBuilder.select().from("ks", "cf");
select.where().and(QueryBuilder.eq("a", 1));
select.where().and(QueryBuilder.in("b", Arrays.asList(1, 1, 1)));
select.setFetchSize(5);

Now we put a distinct method in client side to eliminate the duplicated key, 
but it's better to fix inside Cassandra.


> Duplicated Key in IN clause with a small fetch size will run forever
> 
>
> Key: CASSANDRA-12420
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12420
> Project: Cassandra
>  Issue Type: Bug
> Environment: cassandra 2.1.14, driver 2.1.7.1
>Reporter: ZhaoYang
>Assignee: ZhaoYang
> Fix For: 2.1.16
>
>
> This can be easily reproduced and fetch size is smaller than the correct 
> number of rows.
> A table has 2 partition key, 1 clustering key, 1 column.
> >Select select = QueryBuilder.select().from("ks", "cf");
> >select.where().and(QueryBuilder.eq("a", 1));
> >select.where().and(QueryBuilder.in("b", Arrays.asList(1, 1, 1)));
> >select.setFetchSize(5);
> Now we put a distinct method in client side to eliminate the duplicated key, 
> but it's better to fix inside Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12420) Duplicated Key in IN clause with a small fetch size will run forever

[ZhaoYang (JIRA)]

ZhaoYang created CASSANDRA-12420:


 Summary: Duplicated Key in IN clause with a small fetch size will 
run forever
 Key: CASSANDRA-12420
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12420
 Project: Cassandra
  Issue Type: Bug
 Environment: cassandra 2.1.14, driver 2.1.7.1
Reporter: ZhaoYang
Assignee: ZhaoYang
 Fix For: 2.1.16


This can be easily reproduced and fetch size is smaller than the correct number 
of rows.
Select select = QueryBuilder.select().from("ks", "cf");
select.where().and(QueryBuilder.eq("a", 1));
select.where().and(QueryBuilder.in("b", Arrays.asList(1, 1, 1)));
select.setFetchSize(5);

Now we put a distinct method in client side to eliminate the duplicated key, 
but it's better to fix inside Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12366) Fix compaction throttle

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413836#comment-15413836
 ] 

T Jake Luciani commented on CASSANDRA-12366:


I pushed another change to address the problem and the dtests are now clean.  
Will wait for your confirmation I'm ok to commit.

> Fix compaction throttle
> ---
>
> Key: CASSANDRA-12366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12366
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.x
>
>
> Compaction throttling is broken in the following ways:
>   * It throttles bytes read after being decompressed
>   * Compaction creates multiple scanners which share the rate limiter causing 
> too much throttling
>   * It bears no resemblance to the reported compaction time remaining 
> calculation (Bytes of source sstables processed since start of compaction)
> To fix this we need to simplify the throttling to be only at the 
> CompactionIterator level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12358:
--
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

> Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
> -
>
> Key: CASSANDRA-12358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12358
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.10
>
>
> 2i can be slow to flush for a variety of reasons. Potentially slower than the 
> rate at which Memtables can ingest and flush data. If this occurs the heap 
> fills up with Memtables that are waiting for PostFlush to run.
> This occurs because reclaiming the memory is done before PostFlush runs.
> I will post a branch that has the reclaim memory task run after PostFlush has 
> completed. As far as I can tell this is safe and correct since the memory is 
> committed up until that point.
> It's not clear to me if PostFlush has to bind the Memtables or not. I suspect 
> it does, but I'm not sure if that is a route I should go down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413774#comment-15413774
 ] 

Aleksey Yeschenko commented on CASSANDRA-12358:
---

Committed as 
[4878852fe4aae3516c21fdeafac5c5746a93c31f|https://github.com/apache/cassandra/commit/4878852fe4aae3516c21fdeafac5c5746a93c31f]
 to trunk, thanks.

> Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
> -
>
> Key: CASSANDRA-12358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12358
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.10
>
>
> 2i can be slow to flush for a variety of reasons. Potentially slower than the 
> rate at which Memtables can ingest and flush data. If this occurs the heap 
> fills up with Memtables that are waiting for PostFlush to run.
> This occurs because reclaiming the memory is done before PostFlush runs.
> I will post a branch that has the reclaim memory task run after PostFlush has 
> completed. As far as I can tell this is safe and correct since the memory is 
> committed up until that point.
> It's not clear to me if PostFlush has to bind the Memtables or not. I suspect 
> it does, but I'm not sure if that is a route I should go down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Delay releasing Memtable memory on flush until PostFlush has finished running

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7c60840e9 -> 4878852fe


Delay releasing Memtable memory on flush until PostFlush has finished running

patch by Ariel Weisberg; reviewed by Branimir Lambov for CASSANDRA-12358


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4878852f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4878852f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4878852f

Branch: refs/heads/trunk
Commit: 4878852fe4aae3516c21fdeafac5c5746a93c31f
Parents: 7c60840
Author: Ariel Weisberg 
Authored: Mon Aug 1 20:48:49 2016 -0400
Committer: Aleksey Yeschenko 
Committed: Tue Aug 9 16:58:48 2016 +0100

--
 CHANGES.txt   |  1 +
 .../org/apache/cassandra/db/ColumnFamilyStore.java| 14 --
 2 files changed, 9 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4878852f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5161045..65e8aad 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.10
+ * Delay releasing Memtable memory on flush until PostFlush has finished 
running (CASSANDRA-12358)
  * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
  * Make it possible to compact a given token range (CASSANDRA-10643)
  * Allow updating DynamicEndpointSnitch properties via JMX (CASSANDRA-12179)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4878852f/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 8439111..36f54e7 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -866,9 +866,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 logFlush();
 Flush flush = new Flush(false);
 flushExecutor.execute(flush);
-ListenableFutureTask task = 
ListenableFutureTask.create(flush.postFlush);
-postFlushExecutor.submit(task);
-return task;
+postFlushExecutor.execute(flush.postFlushTask);
+return flush.postFlushTask;
 }
 }
 
@@ -1036,6 +1035,7 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 {
 final OpOrder.Barrier writeBarrier;
 final List memtables = new ArrayList<>();
+final ListenableFutureTask postFlushTask;
 final PostFlush postFlush;
 final boolean truncate;
 
@@ -1078,6 +1078,7 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 // commit log segment position have also completed, i.e. the 
memtables are done and ready to flush
 writeBarrier.issue();
 postFlush = new PostFlush(!truncate, writeBarrier, memtables);
+postFlushTask = ListenableFutureTask.create(postFlush);
 }
 
 public void run()
@@ -1214,14 +1215,14 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 // issue a read barrier for reclaiming the memory, and offload the 
wait to another thread
 final OpOrder.Barrier readBarrier = readOrdering.newBarrier();
 readBarrier.issue();
-reclaimExecutor.execute(new WrappedRunnable()
+postFlushTask.addListener(new WrappedRunnable()
 {
 public void runMayThrow()
 {
 readBarrier.await();
 memtable.setDiscarded();
 }
-});
+}, reclaimExecutor);
 }
 }
 
@@ -2211,7 +2212,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 {
 final Flush flush = new Flush(true);
 flushExecutor.execute(flush);
-return postFlushExecutor.submit(flush.postFlush);
+postFlushExecutor.execute(flush.postFlushTask);
+return flush.postFlushTask;
 }
 }
 

[jira] [Updated] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM

[Branimir Lambov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branimir Lambov updated CASSANDRA-12358:

Status: Ready to Commit  (was: Patch Available)

> Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
> -
>
> Key: CASSANDRA-12358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12358
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.10
>
>
> 2i can be slow to flush for a variety of reasons. Potentially slower than the 
> rate at which Memtables can ingest and flush data. If this occurs the heap 
> fills up with Memtables that are waiting for PostFlush to run.
> This occurs because reclaiming the memory is done before PostFlush runs.
> I will post a branch that has the reclaim memory task run after PostFlush has 
> completed. As far as I can tell this is safe and correct since the memory is 
> committed up until that point.
> It's not clear to me if PostFlush has to bind the Memtables or not. I suspect 
> it does, but I'm not sure if that is a route I should go down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413756#comment-15413756
 ] 

Branimir Lambov commented on CASSANDRA-12358:
-

The dtest didn't actually run correctly (just a fraction of the tests ran). The 
next run is fine, though (one fail, same as trunk).

LGTM

> Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
> -
>
> Key: CASSANDRA-12358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12358
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.10
>
>
> 2i can be slow to flush for a variety of reasons. Potentially slower than the 
> rate at which Memtables can ingest and flush data. If this occurs the heap 
> fills up with Memtables that are waiting for PostFlush to run.
> This occurs because reclaiming the memory is done before PostFlush runs.
> I will post a branch that has the reclaim memory task run after PostFlush has 
> completed. As far as I can tell this is safe and correct since the memory is 
> committed up until that point.
> It's not clear to me if PostFlush has to bind the Memtables or not. I suspect 
> it does, but I'm not sure if that is a route I should go down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11726) IndexOutOfBoundsException when selecting (distinct) row ids from counter table.

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11726:
--
   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.9
Reproduced In: 3.7, 3.5  (was: 3.5, 3.7)
   Status: Resolved  (was: Patch Available)

> IndexOutOfBoundsException when selecting (distinct) row ids from counter 
> table.
> ---
>
> Key: CASSANDRA-11726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11726
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: C* 3.5, cluster of 4 nodes.
>Reporter: Jaroslav Kamenik
>Assignee: Sylvain Lebresne
> Fix For: 3.9
>
>
> I have simple table containing counters:
> {code}
> CREATE TABLE tablename (
> object_id ascii,
> counter_id ascii,
> count counter,
> PRIMARY KEY (object_id, counter_id)
> ) WITH CLUSTERING ORDER BY (counter_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'enabled': 'false'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> Counters are often inc/decreased, whole rows are queried, deleted sometimes.
> After some time I tried to query all object_ids, but it failed with:
> {code}
> cqlsh:woc> consistency quorum;
> cqlsh:woc> select object_id from tablename;
> ServerError:  message="java.lang.IndexOutOfBoundsException">
> {code}
> select * from ..., select where .., updates works well..
> With consistency one it works sometimes, so it seems something is broken at 
> one server, but I tried to repair table there and it did not help. 
> Whole exception from server log:
> {code}
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_73]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_73]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:591)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:549)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:526) 
> ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> 

[jira] [Commented] (CASSANDRA-11726) IndexOutOfBoundsException when selecting (distinct) row ids from counter table.

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413748#comment-15413748
 ] 

Aleksey Yeschenko commented on CASSANDRA-11726:
---

LGTM. Committed as 
[ee609411a8e154255812b157788a79bbdf076566|https://github.com/apache/cassandra/commit/ee609411a8e154255812b157788a79bbdf076566]
 to 3.9 and trunk.

> IndexOutOfBoundsException when selecting (distinct) row ids from counter 
> table.
> ---
>
> Key: CASSANDRA-11726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11726
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: C* 3.5, cluster of 4 nodes.
>Reporter: Jaroslav Kamenik
>Assignee: Sylvain Lebresne
> Fix For: 3.x
>
>
> I have simple table containing counters:
> {code}
> CREATE TABLE tablename (
> object_id ascii,
> counter_id ascii,
> count counter,
> PRIMARY KEY (object_id, counter_id)
> ) WITH CLUSTERING ORDER BY (counter_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'enabled': 'false'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> Counters are often inc/decreased, whole rows are queried, deleted sometimes.
> After some time I tried to query all object_ids, but it failed with:
> {code}
> cqlsh:woc> consistency quorum;
> cqlsh:woc> select object_id from tablename;
> ServerError:  message="java.lang.IndexOutOfBoundsException">
> {code}
> select * from ..., select where .., updates works well..
> With consistency one it works sometimes, so it seems something is broken at 
> one server, but I tried to repair table there and it did not help. 
> Whole exception from server log:
> {code}
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_73]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_73]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:591)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:549)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:526) 
> ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.5.jar:3.5]
> at 
> 

[2/3] cassandra git commit: Fix value skipping with counter columns

[aleksey]

Fix value skipping with counter columns

patch by Sylvain Lebresne; reviewed by Aleksey Yeschenko for
CASSANDRA-11726


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ee609411
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ee609411
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ee609411

Branch: refs/heads/trunk
Commit: ee609411a8e154255812b157788a79bbdf076566
Parents: 02ebf87
Author: Sylvain Lebresne 
Authored: Thu Aug 4 17:54:03 2016 +0200
Committer: Aleksey Yeschenko 
Committed: Tue Aug 9 16:33:10 2016 +0100

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/Conflicts.java |  9 +++
 .../cql3/validation/entities/CountersTest.java  | 27 
 3 files changed, 37 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee609411/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c5c221..b7bbf72 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Fix value skipping with counter columns (CASSANDRA-11726)
  * Fix nodetool tablestats miss SSTable count (CASSANDRA-12205)
  * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
  * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee609411/src/java/org/apache/cassandra/db/Conflicts.java
--
diff --git a/src/java/org/apache/cassandra/db/Conflicts.java 
b/src/java/org/apache/cassandra/db/Conflicts.java
index fa0e819..9e8bd9a 100644
--- a/src/java/org/apache/cassandra/db/Conflicts.java
+++ b/src/java/org/apache/cassandra/db/Conflicts.java
@@ -68,6 +68,15 @@ public abstract class Conflicts
 if (!rightLive)
 return Resolution.RIGHT_WINS;
 
+// Handle empty values. Counters can't truly have empty values, but we 
can have a counter cell that temporarily
+// has one on read if the column for the cell is not queried by the 
user due to the optimization of #10657. We
+// thus need to handle this (see #11726 too).
+if (!leftValue.hasRemaining())
+return rightValue.hasRemaining() || leftTimestamp > rightTimestamp 
? Resolution.LEFT_WINS : Resolution.RIGHT_WINS;
+
+if (!rightValue.hasRemaining())
+return Resolution.RIGHT_WINS;
+
 return Resolution.MERGE;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee609411/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
index 33b4a4f..b1cd0bb 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
@@ -196,4 +196,31 @@ public class CountersTest extends CQLTester
 assertInvalidThrowMessage("counter type is not supported for PRIMARY 
KEY part a",
   InvalidRequestException.class, 
String.format("CREATE TABLE %s.%s (a counter, b int, PRIMARY KEY (b, a)) WITH 
CLUSTERING ORDER BY (a desc);", KEYSPACE, createTableName()));
 }
+
+/**
+ * Test for the bug of #11726.
+ */
+@Test
+public void testCounterAndColumnSelection() throws Throwable
+{
+for (String compactStorageClause : new String[]{ "", " WITH COMPACT 
STORAGE" })
+{
+createTable("CREATE TABLE %s (k int PRIMARY KEY, c counter)" + 
compactStorageClause);
+
+// Flush 2 updates in different sstable so that the following 
select does a merge, which is what triggers
+// the problem from #11726
+
+execute("UPDATE %s SET c = c + ? WHERE k = ?", 1L, 0);
+
+flush();
+
+execute("UPDATE %s SET c = c + ? WHERE k = ?", 1L, 0);
+
+flush();
+
+// Querying, but not including the counter. Pre-CASSANDRA-11726, 
this made us query the counter but include
+// it's value, which broke at merge (post-CASSANDRA-11726 are 
special cases to never skip values).
+assertRows(execute("SELECT k FROM %s"), row(0));
+}
+}
 }

[3/3] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[aleksey]

Merge branch 'cassandra-3.9' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c60840e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c60840e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c60840e

Branch: refs/heads/trunk
Commit: 7c60840e945823b8ad9aecfd68b290a32301b7e6
Parents: 2c4b301 ee60941
Author: Aleksey Yeschenko 
Authored: Tue Aug 9 16:36:28 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Tue Aug 9 16:36:28 2016 +0100

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/Conflicts.java |  9 +++
 .../cql3/validation/entities/CountersTest.java  | 27 
 3 files changed, 37 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c60840e/CHANGES.txt
--
diff --cc CHANGES.txt
index 097e9c1,b7bbf72..5161045
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,39 -1,5 +1,40 @@@
 +3.10
 + * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
 + * Make it possible to compact a given token range (CASSANDRA-10643)
 + * Allow updating DynamicEndpointSnitch properties via JMX (CASSANDRA-12179)
 + * Collect metrics on queries by consistency level (CASSANDRA-7384)
 + * Add support for GROUP BY to SELECT statement (CASSANDRA-10707)
 + * Deprecate memtable_cleanup_threshold and update default for 
memtable_flush_writers (CASSANDRA-12228)
 + * Upgrade to OHC 0.4.4 (CASSANDRA-12133)
 + * Add version command to cassandra-stress (CASSANDRA-12258)
 + * Create compaction-stress tool (CASSANDRA-11844)
 + * Garbage-collecting compaction operation and schema option (CASSANDRA-7019)
 + * Add schema to snapshot manifest, add USING TIMESTAMP clause to ALTER TABLE 
statements (CASSANDRA-7190)
 + * Add beta protocol flag for v5 native protocol (CASSANDRA-12142)
 + * Support filtering on non-PRIMARY KEY columns in the CREATE
 +   MATERIALIZED VIEW statement's WHERE clause (CASSANDRA-10368)
 + * Unify STDOUT and SYSTEMLOG logback format (CASSANDRA-12004)
 + * COPY FROM should raise error for non-existing input files (CASSANDRA-12174)
 + * Faster write path (CASSANDRA-12269)
 + * Option to leave omitted columns in INSERT JSON unset (CASSANDRA-11424)
 + * Support json/yaml output in nodetool tpstats (CASSANDRA-12035)
 + * Expose metrics for successful/failed authentication attempts 
(CASSANDRA-10635)
 + * Prepend snapshot name with "truncated" or "dropped" when a snapshot
 +   is taken before truncating or dropping a table (CASSANDRA-12178)
 + * Optimize RestrictionSet (CASSANDRA-12153)
 + * cqlsh does not automatically downgrade CQL version (CASSANDRA-12150)
 + * Omit (de)serialization of state variable in UDAs (CASSANDRA-9613)
 + * Create a system table to expose prepared statements (CASSANDRA-8831)
 + * Reuse DataOutputBuffer from ColumnIndex (CASSANDRA-11970)
 + * Remove DatabaseDescriptor dependency from SegmentedFile (CASSANDRA-11580)
 + * Add supplied username to authentication error messages (CASSANDRA-12076)
 + * Remove pre-startup check for open JMX port (CASSANDRA-12074)
 + * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738)
 +
 +
  3.9
+  * Fix value skipping with counter columns (CASSANDRA-11726)
 + * Restore resumable hints delivery (CASSANDRA-11960)
   * Fix nodetool tablestats miss SSTable count (CASSANDRA-12205)
   * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
   * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c60840e/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
--

[1/3] cassandra git commit: Fix value skipping with counter columns

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.9 02ebf875b -> ee609411a
  refs/heads/trunk 2c4b30164 -> 7c60840e9


Fix value skipping with counter columns

patch by Sylvain Lebresne; reviewed by Aleksey Yeschenko for
CASSANDRA-11726


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ee609411
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ee609411
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ee609411

Branch: refs/heads/cassandra-3.9
Commit: ee609411a8e154255812b157788a79bbdf076566
Parents: 02ebf87
Author: Sylvain Lebresne 
Authored: Thu Aug 4 17:54:03 2016 +0200
Committer: Aleksey Yeschenko 
Committed: Tue Aug 9 16:33:10 2016 +0100

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/Conflicts.java |  9 +++
 .../cql3/validation/entities/CountersTest.java  | 27 
 3 files changed, 37 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee609411/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c5c221..b7bbf72 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Fix value skipping with counter columns (CASSANDRA-11726)
  * Fix nodetool tablestats miss SSTable count (CASSANDRA-12205)
  * Fixed flacky SSTablesIteratedTest (CASSANDRA-12282)
  * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee609411/src/java/org/apache/cassandra/db/Conflicts.java
--
diff --git a/src/java/org/apache/cassandra/db/Conflicts.java 
b/src/java/org/apache/cassandra/db/Conflicts.java
index fa0e819..9e8bd9a 100644
--- a/src/java/org/apache/cassandra/db/Conflicts.java
+++ b/src/java/org/apache/cassandra/db/Conflicts.java
@@ -68,6 +68,15 @@ public abstract class Conflicts
 if (!rightLive)
 return Resolution.RIGHT_WINS;
 
+// Handle empty values. Counters can't truly have empty values, but we 
can have a counter cell that temporarily
+// has one on read if the column for the cell is not queried by the 
user due to the optimization of #10657. We
+// thus need to handle this (see #11726 too).
+if (!leftValue.hasRemaining())
+return rightValue.hasRemaining() || leftTimestamp > rightTimestamp 
? Resolution.LEFT_WINS : Resolution.RIGHT_WINS;
+
+if (!rightValue.hasRemaining())
+return Resolution.RIGHT_WINS;
+
 return Resolution.MERGE;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee609411/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
index 33b4a4f..b1cd0bb 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/CountersTest.java
@@ -196,4 +196,31 @@ public class CountersTest extends CQLTester
 assertInvalidThrowMessage("counter type is not supported for PRIMARY 
KEY part a",
   InvalidRequestException.class, 
String.format("CREATE TABLE %s.%s (a counter, b int, PRIMARY KEY (b, a)) WITH 
CLUSTERING ORDER BY (a desc);", KEYSPACE, createTableName()));
 }
+
+/**
+ * Test for the bug of #11726.
+ */
+@Test
+public void testCounterAndColumnSelection() throws Throwable
+{
+for (String compactStorageClause : new String[]{ "", " WITH COMPACT 
STORAGE" })
+{
+createTable("CREATE TABLE %s (k int PRIMARY KEY, c counter)" + 
compactStorageClause);
+
+// Flush 2 updates in different sstable so that the following 
select does a merge, which is what triggers
+// the problem from #11726
+
+execute("UPDATE %s SET c = c + ? WHERE k = ?", 1L, 0);
+
+flush();
+
+execute("UPDATE %s SET c = c + ? WHERE k = ?", 1L, 0);
+
+flush();
+
+// Querying, but not including the counter. Pre-CASSANDRA-11726, 
this made us query the counter but include
+// it's value, which broke at merge (post-CASSANDRA-11726 are 
special cases to never skip values).
+assertRows(execute("SELECT k FROM %s"), row(0));
+}
+}
 }

[jira] [Commented] (CASSANDRA-11752) histograms/metrics in 2.2 do not appear recency biased

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413722#comment-15413722
 ] 

T Jake Luciani commented on CASSANDRA-11752:


Great thanks for this verification:

I've kicked off the tests:

2.2
[branch|https://github.com/tjake/cassandra/tree/11752-2.2]
[testall|http://cassci.datastax.com/job/tjake-11752-2.2-testall/]
[dtest|http://cassci.datastax.com/job/tjake-11752-2.2-dtest/]

3.0
[branch|https://github.com/tjake/cassandra/tree/11752-3.0]
[testall|http://cassci.datastax.com/job/tjake-11752-3.0-testall/]
[dtest|http://cassci.datastax.com/job/tjake-11752-3.0-dtest/]


3.9
[branch|https://github.com/tjake/cassandra/tree/11752-3.9]
[testall|http://cassci.datastax.com/job/tjake-11752-3.9-testall/]
[dtest|http://cassci.datastax.com/job/tjake-11752-3.9-dtest/]


> histograms/metrics in 2.2 do not appear recency biased
> --
>
> Key: CASSANDRA-11752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11752
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Burroughs
>Assignee: Per Otterström
>  Labels: metrics
> Fix For: 2.2.8
>
> Attachments: 11752-2.2-v2.txt, 11752-2.2.txt, boost-metrics.png, 
> c-jconsole-comparison.png, c-metrics.png, default-histogram.png, 
> server-patch-v2.png
>
>
> In addition to upgrading to metrics3, CASSANDRA-5657 switched to using  a 
> custom histogram implementation.  After upgrading to Cassandra 2.2 
> histograms/timer metrics are not suspiciously flat.  To be useful for 
> graphing and alerting metrics need to be biased towards recent events.
> I have attached images that I think illustrate this.
>  * The first two are a comparison between latency observed by a C* 2.2 (us) 
> cluster shoring very flat lines and a client (using metrics 2.2.0, ms) 
> showing server performance problems.  We can't rule out with total certainty 
> that something else isn't the cause (that's why we measure from both the 
> client & server) but they very rarely disagree.
>  * The 3rd image compares jconsole viewing of metrics on a 2.2 and 2.1 
> cluster over several minutes.  Not a single digit changed on the 2.2 cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12152) Unknown exception caught while attempting to update MaterializedView: AssertionError: Flags = 128

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413712#comment-15413712
 ] 

ZhaoYang commented on CASSANDRA-12152:
--

it seems that the row flag is being read as 'static', but Cassandra is 
expecting 'Row' or 'Tombstone' type.

> Unknown exception caught while attempting to update MaterializedView: 
> AssertionError: Flags = 128
> -
>
> Key: CASSANDRA-12152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12152
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Nilson Pontello
>
> I have a single DC with 3 cassandra nodes. After a restart today, none of 
> them were capable of processing the commitlog while starting up. The 
> exception doesn't contains enough information about what is going on, please 
> check bellow:
> {code}
> ERROR [SharedPool-Worker-21] 2016-07-08 12:42:12,866 Keyspace.java:521 - 
> Unknown exception caught while attempting to update MaterializedView! 
> data_monitor.user_timeline
> java.lang.AssertionError: Flags = 128
>  at 
> org.apache.cassandra.db.ClusteringPrefix$Deserializer.prepare(ClusteringPrefix.java:421)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.prepareNext(UnfilteredDeserializer.java:172)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.hasNext(UnfilteredDeserializer.java:153)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.handlePreSliceData(SSTableIterator.java:96)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:141)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:637)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:586)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:463)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325)
>  ~[apache-cassandra-3.5.jar:3.5]
>  at 
> 

[jira] [Updated] (CASSANDRA-12407) dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-12407:

Reviewer: Stefania
  Status: Patch Available  (was: Open)

https://github.com/riptano/cassandra-dtest/pull/1195

Given that Tyler was once okay with skipping on 2.1, and you seem to be as 
well, that is preferable to me.

> dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test
> 
>
> Key: CASSANDRA-12407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12407
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/381/testReport/cql_tracing_test/TestCqlTracing/tracing_simple_test
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 102, in 
> tracing_simple_test
> self.trace(session)
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 74, in 
> trace
> self.assertIn('/127.0.0.1', out)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> '\'/127.0.0.1\' not found in "Consistency level set to ALL.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12419) dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson resolved CASSANDRA-12419.
-
Resolution: Fixed

Fixed in dtest commit e294e1d3b99

> dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
> -
>
> Key: CASSANDRA-12419
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12419
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: Philip Thompson
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/31/testReport/bootstrap_test/TestBootstrap/local_quorum_bootstrap_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12419) dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson reassigned CASSANDRA-12419:
---

Assignee: Philip Thompson  (was: DS Test Eng)

> dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
> -
>
> Key: CASSANDRA-12419
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12419
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: Philip Thompson
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/31/testReport/bootstrap_test/TestBootstrap/local_quorum_bootstrap_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12404) ColumnFamilyStore.getIfExists

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12404:
--
Component/s: (was: Packaging)

> ColumnFamilyStore.getIfExists
> -
>
> Key: CASSANDRA-12404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12404
> Project: Cassandra
>  Issue Type: Bug
>Reporter: stone
> Attachments: heartbeat.PNG
>
>
> ColumnFamilyStore.getIfExists("ks","cf")
> when it does not exist ks.cf,it should return null,but now there is a 
> Nullpointexception.
> see heartbeat.PNG



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12404) ColumnFamilyStore.getIfExists

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12404:
--
Priority: Minor  (was: Major)

> ColumnFamilyStore.getIfExists
> -
>
> Key: CASSANDRA-12404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12404
> Project: Cassandra
>  Issue Type: Bug
>Reporter: stone
>Priority: Minor
> Attachments: heartbeat.PNG
>
>
> ColumnFamilyStore.getIfExists("ks","cf")
> when it does not exist ks.cf,it should return null,but now there is a 
> Nullpointexception.
> see heartbeat.PNG



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12380) Cassandra doesn't start with IndexOutOfBoundsException exception

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413582#comment-15413582
 ] 

Aleksey Yeschenko commented on CASSANDRA-12380:
---

Might be regular disk corruption. Have you tried scrubbing it?

> Cassandra doesn't start with IndexOutOfBoundsException exception
> 
>
> Key: CASSANDRA-12380
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12380
> Project: Cassandra
>  Issue Type: Bug
> Environment: Version 2.2.4
>Reporter: Artem Rokhin
>
> {code}
> 2016-08-02T10:31:35,260 [CompactionExecutor:1] ERROR 
> o.a.c.service.CassandraDaemon - Exception in thread 
> Thread[CompactionExecutor:1,1,main]
> org.apache.cassandra.io.FSReadError: java.lang.IndexOutOfBoundsException
>   at 
> org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:358)
>  ~[apache-cassandra-clientutil-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:359) 
> ~[apache-cassandra-clientutil-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:338)
>  ~[apache-cassandra-clientutil-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:365)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
>   at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:169)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
>   at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
>   at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
>   at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
>   at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:125)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:136)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:116)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.append(MaxSSTableSizeWriter.java:67)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:184)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:247)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_73]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_73]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_73]
>   at 
>