[jira] [Updated] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14110:
---
Status: Patch Available  (was: Open)

> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> 

[jira] [Commented] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288752#comment-16288752
 ] 

Jay Zhuang commented on CASSANDRA-14110:


The patch is updated.

> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] 

[jira] [Updated] (CASSANDRA-14099) LCS ordering of sstables by timestamp is inverted

[Vincent White (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vincent White updated CASSANDRA-14099:
--
Status: Patch Available  (was: Open)

I've created a patch that splits in this comparator in two as a way to maybe 
help avoid this confusion in the future. Now that this is split into two I'm 
not sure if a unit test for ageSortedSSTables (or the comparators themselves) 
would be required? I have included a unit test for ageSortedSSTables on my 
3.0.x branch, not sure if it's worth making ageSortedSStable() Public just for 
this but I didn't see anywhere else where its behaviour was visible.

[3.0 patch | 
https://github.com/vincewhite/cassandra/commits/14099_timestamp_comparators_30]
[3.0 utest | 
https://github.com/vincewhite/cassandra/commit/5ab1ff36a28b41039bd93de7d47b4131e1c2dfaa]
[3.x patch | 
https://github.com/vincewhite/cassandra/commits/14099_timestamp_comparators_311]
[trunk patch | 
https://github.com/vincewhite/cassandra/commits/14099_timestamp_comparators_trunk]



> LCS ordering of sstables by timestamp is inverted
> -
>
> Key: CASSANDRA-14099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14099
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> In CASSANDRA-14010 we discovered that CASSANDRA-13776 broke sstable ordering 
> by timestamp (inverted it accidentally). Investigating that revealed that the 
> comparator was expecting newest-to-oldest for read command, but LCS expects 
> oldest-to-newest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Added additional details to security documetation discussing attack surface.

[zznate]

Repository: cassandra
Updated Branches:
  refs/heads/trunk dc9bb8df0 -> 0d8199bab


Added additional details to security documetation discussing attack surface.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d8199ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d8199ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d8199ba

Branch: refs/heads/trunk
Commit: 0d8199bab25d08c8b08adb3803dd7825894c5306
Parents: dc9bb8d
Author: Nate McCall 
Authored: Wed Dec 13 14:48:17 2017 +1300
Committer: Nate McCall 
Committed: Wed Dec 13 14:48:17 2017 +1300

--
 doc/source/operating/security.rst | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8199ba/doc/source/operating/security.rst
--
diff --git a/doc/source/operating/security.rst 
b/doc/source/operating/security.rst
index dfcd9e6..212a25e 100644
--- a/doc/source/operating/security.rst
+++ b/doc/source/operating/security.rst
@@ -18,13 +18,25 @@
 
 Security
 
-
 There are three main components to the security features provided by Cassandra:
 
 - TLS/SSL encryption for client and inter-node communication
 - Client authentication
 - Authorization
 
+By default, these features are disabled as Cassandra is configured to easily 
find and be found by other members of a
+cluster. In other words, an out-of-the-box Cassandra installation presents a 
large attack surface for a bad actor.
+Possible attack vectors include:
+
+- Crafted internode messages to insert users into authentication schema
+- Crafted internode messages to truncate or drop schema
+- Use of tools such as ``sstableloader`` to overwrite ``system_auth`` tables 
+- Attaching to the cluster directly to capture write traffic
+
+Correct configuration of all three security components should negate theses 
vectors. Therefore, understanding Cassandra's
+security features is crucial to configuring your cluster to meet your security 
needs.
+
+
 TLS/SSL Encryption
 ^^
 Cassandra provides secure communication between a client machine and a 
database cluster and between nodes within a


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14106) utest failed: DistributionSequenceTest.setSeed() and simpleSequence()

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288569#comment-16288569
 ] 

Blake Eggleston edited comment on CASSANDRA-14106 at 12/13/17 1:17 AM:
---

If stress works properly, then it's ok with me.

edit: actually, taking another look at the code, [~jay.zhuang]'s change is the 
right thing to do here


was (Author: bdeggleston):
If stress works properly, then it's ok with me.

> utest failed: DistributionSequenceTest.setSeed() and simpleSequence()
> -
>
> Key: CASSANDRA-14106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> To reproduce:
> {noformat}
> $ ant stress-test -Dtest.name=DistributionSequenceTest
> {noformat}
> {noformat}
> stress-test:
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest Tests run: 4, 
> Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
> [junit]
> [junit] Testcase: 
> simpleSequence(org.apache.cassandra.stress.generate.DistributionSequenceTest):
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.simpleSequence(DistributionSequenceTest.java:37)
> [junit]
> [junit]
> [junit] Testcase: 
> setSeed(org.apache.cassandra.stress.generate.DistributionSequenceTest):   
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.setSeed(DistributionSequenceTest.java:111)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.stress.generate.DistributionSequenceTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14106) utest failed: DistributionSequenceTest.setSeed() and simpleSequence()

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288569#comment-16288569
 ] 

Blake Eggleston commented on CASSANDRA-14106:
-

If stress works properly, then it's ok with me.

> utest failed: DistributionSequenceTest.setSeed() and simpleSequence()
> -
>
> Key: CASSANDRA-14106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> To reproduce:
> {noformat}
> $ ant stress-test -Dtest.name=DistributionSequenceTest
> {noformat}
> {noformat}
> stress-test:
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest Tests run: 4, 
> Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
> [junit]
> [junit] Testcase: 
> simpleSequence(org.apache.cassandra.stress.generate.DistributionSequenceTest):
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.simpleSequence(DistributionSequenceTest.java:37)
> [junit]
> [junit]
> [junit] Testcase: 
> setSeed(org.apache.cassandra.stress.generate.DistributionSequenceTest):   
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.setSeed(DistributionSequenceTest.java:111)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.stress.generate.DistributionSequenceTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14106) utest failed: DistributionSequenceTest.setSeed() and simpleSequence()

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288419#comment-16288419
 ] 

Jay Zhuang edited comment on CASSANDRA-14106 at 12/12/17 11:32 PM:
---

Hi [~bdeggleston], after changing the {{double d}} to {{float d}}, it won't 
break stress. In the patch, I also added an unittest to reproduce the issue 
introduced by {{lgmt.com}} change.

The problem with {{double}} is: in the last round, the value of {{d}} would be 
{{1....004}} instead of {{1.0}}. Which causes the exception 
{{OutOfRangeException: 1 out of [0, 1] range}} in cassandra-stress gaussian 
distribution.


was (Author: jay.zhuang):
Hi [~bdeggleston], after changing the {{double d}} to {{float d}}, it won't 
break stress. In the patch, I also added an unittest to reproduce the issue 
introduced by {{lgmt.com}} change.

> utest failed: DistributionSequenceTest.setSeed() and simpleSequence()
> -
>
> Key: CASSANDRA-14106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> To reproduce:
> {noformat}
> $ ant stress-test -Dtest.name=DistributionSequenceTest
> {noformat}
> {noformat}
> stress-test:
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest Tests run: 4, 
> Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
> [junit]
> [junit] Testcase: 
> simpleSequence(org.apache.cassandra.stress.generate.DistributionSequenceTest):
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.simpleSequence(DistributionSequenceTest.java:37)
> [junit]
> [junit]
> [junit] Testcase: 
> setSeed(org.apache.cassandra.stress.generate.DistributionSequenceTest):   
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.setSeed(DistributionSequenceTest.java:111)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.stress.generate.DistributionSequenceTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14110:
---
Status: Open  (was: Patch Available)

Canceling patch, the test is still failed.

> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> 

[jira] [Commented] (CASSANDRA-14106) utest failed: DistributionSequenceTest.setSeed() and simpleSequence()

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288419#comment-16288419
 ] 

Jay Zhuang commented on CASSANDRA-14106:


Hi [~bdeggleston], after changing the {{double d}} to {{float d}}, it won't 
break stress. In the patch, I also added an unittest to reproduce the issue 
introduced by {{lgmt.com}} change.

> utest failed: DistributionSequenceTest.setSeed() and simpleSequence()
> -
>
> Key: CASSANDRA-14106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> To reproduce:
> {noformat}
> $ ant stress-test -Dtest.name=DistributionSequenceTest
> {noformat}
> {noformat}
> stress-test:
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest Tests run: 4, 
> Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
> [junit]
> [junit] Testcase: 
> simpleSequence(org.apache.cassandra.stress.generate.DistributionSequenceTest):
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.simpleSequence(DistributionSequenceTest.java:37)
> [junit]
> [junit]
> [junit] Testcase: 
> setSeed(org.apache.cassandra.stress.generate.DistributionSequenceTest):   
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.setSeed(DistributionSequenceTest.java:111)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.stress.generate.DistributionSequenceTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13928) Remove initialDirectories from CFS

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288413#comment-16288413
 ] 

Paulo Motta commented on CASSANDRA-13928:
-

LGTM, can you just rebase (now that CASSANDRA-13948 is in) and submit a new 
test run? Feel free to commit when CI is clean.

> Remove initialDirectories from CFS
> --
>
> Key: CASSANDRA-13928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13928
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> The initialDirectories added in CASSANDRA-8671 is quite confusing and I don't 
> think it is needed anymore, it should be removed



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13928) Remove initialDirectories from CFS

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-13928:

Status: Ready to Commit  (was: Patch Available)

> Remove initialDirectories from CFS
> --
>
> Key: CASSANDRA-13928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13928
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> The initialDirectories added in CASSANDRA-8671 is quite confusing and I don't 
> think it is needed anymore, it should be removed



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14106) utest failed: DistributionSequenceTest.setSeed() and simpleSequence()

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288408#comment-16288408
 ] 

Blake Eggleston commented on CASSANDRA-14106:
-

undoing that change is going to break stress. This will no longer work: 
{{tools/bin/cassandra-stress user profile=tools/cqlstress-example.yaml 
'ops(insert=1,simple=1)' n=1000 cl=ONE -mode native cql3 -rate threads=100 
fixed=1000/s}}

However, as [~jasonstack] pointed out, my change only fixed part of the 
problem, since I didn't fix the denominator in the return statement. I was just 
going to just ninja fix that, but it's worth seeing if that would fix this 
problem and doing that instead.

In the future, we might want to reject any one time static analysis fixes that 
aren't going to become part of the build process.

> utest failed: DistributionSequenceTest.setSeed() and simpleSequence()
> -
>
> Key: CASSANDRA-14106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> To reproduce:
> {noformat}
> $ ant stress-test -Dtest.name=DistributionSequenceTest
> {noformat}
> {noformat}
> stress-test:
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest Tests run: 4, 
> Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
> [junit]
> [junit] Testcase: 
> simpleSequence(org.apache.cassandra.stress.generate.DistributionSequenceTest):
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.simpleSequence(DistributionSequenceTest.java:37)
> [junit]
> [junit]
> [junit] Testcase: 
> setSeed(org.apache.cassandra.stress.generate.DistributionSequenceTest):   
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.setSeed(DistributionSequenceTest.java:111)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.stress.generate.DistributionSequenceTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction

[Dan Kinder (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288381#comment-16288381
 ] 

Dan Kinder commented on CASSANDRA-13973:


Yep [~jjirsa] that did the trick.

> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Dan Kinder
>Assignee: Jeff Jirsa
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try 
> to run upgradesstables, most of them upgrade fine but I see the exception 
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I 
> assumed it is not identical. The various nodes this happens on all give the 
> same trace.
> Might be notable that this is an analytics cluster with some large 
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at 
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at 
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at 
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288359#comment-16288359
 ] 

Jay Zhuang edited comment on CASSANDRA-14110 at 12/12/17 10:15 PM:
---

I'm able to reproduce the problem on a Linux machine. The patch adds try-catch 
for {{OperationTimedOutException}}, so the test could retry, please review:
| Branch | uTest |
| [14110|https://github.com/cooldoger/cassandra/tree/14110] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14110.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14110]
 |

Also, these trivial changes (CASSANDRA-14111, CASSANDRA-13077) are useful for 
debugging such issue, please review too :)




was (Author: jay.zhuang):
I'm able to reproduce the problem on a Linux machine. The patch adds try-catch 
for {{OperationTimedOutException}}, so the test could retry, please review:
| Branch | uTest |
| [14110|https://github.com/cooldoger/cassandra/tree/14110] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14110.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14110]
 |

Also, these trivial changes (CASSANDRA-14111, CASSANDRA-13077) could be useful 
for debugging such issue.



> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] 

[jira] [Commented] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288359#comment-16288359
 ] 

Jay Zhuang commented on CASSANDRA-14110:


I'm able to reproduce the problem on a Linux machine. The patch adds try-catch 
for {{OperationTimedOutException}}, so the test could retry, please review:
| Branch | uTest |
| [14110|https://github.com/cooldoger/cassandra/tree/14110] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14110.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14110]
 |

Also, these trivial changes (CASSANDRA-14111, CASSANDRA-13077) could be useful 
for debugging such issue.



> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> 

[jira] [Commented] (CASSANDRA-14111) Add long-testsome build target to run specific test

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288355#comment-16288355
 ] 

Jay Zhuang commented on CASSANDRA-14111:


Similar to
{{$ ant testsome}} for {{$ ant test}},
{{$ ant long-testsome}} is for {{$ ant long-test}}, I find it's useful for 
long-test debugging:

| Branch | uTest |
| [14111|https://github.com/cooldoger/cassandra/tree/14111] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14111.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14111]
 |

> Add long-testsome build target to run specific test
> ---
>
> Key: CASSANDRA-14111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14111
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14111) Add long-testsome build target to run specific test

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14111:
---
Status: Patch Available  (was: Open)

> Add long-testsome build target to run specific test
> ---
>
> Key: CASSANDRA-14111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14111
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14111) Add long-testsome build target to run specific test

[Jay Zhuang (JIRA)]

Jay Zhuang created CASSANDRA-14111:
--

 Summary: Add long-testsome build target to run specific test
 Key: CASSANDRA-14111
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14111
 Project: Cassandra
  Issue Type: Improvement
  Components: Build
Reporter: Jay Zhuang
Assignee: Jay Zhuang
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14110:
---
Status: Patch Available  (was: Open)

> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> 

[jira] [Assigned] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang reassigned CASSANDRA-14110:
--

Assignee: Jay Zhuang

> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> 

[jira] [Updated] (CASSANDRA-14108) Improve commit log chain marker updating

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-14108:
---
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.11.2
   3.0.16

> Improve commit log chain marker updating
> 
>
> Key: CASSANDRA-14108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 3.0.16, 3.11.2, 4.x
>
>
> CASSANDRA-13987 addressed the commit log behavior change that was introduced 
> with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
> review and found a bug as well as having some concerns about the 
> configuration. He and I discussed offline, and agreed on some improvements. 
> Instead of requiring users to configure a deep, dark implementation detail 
> like the commit log chained markers (via {{commitlog_marker_period_in_ms}} in 
> the yaml), we decided it is best to eliminate thew configuration and always 
> update the chained markers (when in periodic mode). 
> The bug [~aweisberg] found was when the chained marker update is not a value 
> that evenly divides into the periodic sync mode value, we would not sync in 
> an expected manner. For example if the marker interval is 9 seconds, and the 
> sync interval is 10 seconds, we would update the markers at time9, but we 
> would then sleep for another 9 seconds, and when we wake up at time18, it is 
> then that we flush - 8 seconds later than we should have. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13801) CompactionManager sometimes wrongly determines that a background compaction is running for a particular table

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-13801:

   Resolution: Fixed
Fix Version/s: 4.0
   3.11.2
   3.0.16
   2.2.12
   Status: Resolved  (was: Patch Available)

Good catch, patch and tests LGTM! I looked the results of the dtests on 
internal CI and they look good (unrelated failures), can you just attach the 
screen shots so they are public?

I will commit this to 2.2 even though it is in critical fixes mode because it's 
pretty simple/safe and the only way to solve it is to restart the node if it 
ever gets into this nasty state.

I added a comment on 
[CompactionManager.submitBackground|https://github.com/apache/cassandra/commit/35e6d61361e699908d73c277da7d9ac3390f6e5d#diff-d4e3b82e9bebfd2cb466b4a30af07fa4R159]
 extracted from CASSANDRA-4310 to explain the reasoning behind keeping track of 
compacting cfs.

Committed to 2.2 as {{35e6d61361e699908d73c277da7d9ac3390f6e5d}} and merged up 
to cassandra-3.0, cassandra-3.11 and trunk. Thanks!

> CompactionManager sometimes wrongly determines that a background compaction 
> is running for a particular table
> -
>
> Key: CASSANDRA-13801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13801
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Dimitar Dimitrov
>Assignee: Dimitar Dimitrov
>Priority: Minor
> Fix For: 2.2.12, 3.0.16, 3.11.2, 4.0
>
> Attachments: c13801-2.2-testall.png, c13801-3.0-testall.png, 
> c13801-3.11-testall.png, c13801-trunk-testall.png
>
>
> Sometimes after writing different rows to a table, then doing a blocking 
> flush, if you alter the compaction strategy, then run background compaction 
> and wait for it to finish, {{CompactionManager}} may decide that there's an 
> ongoing compaction for that same table.
> This may happen even though logs don't indicate that to be the case 
> (compaction may still be running for system_schema tables).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[02/10] cassandra git commit: Fix race that prevents submitting compaction for a table when executor is full

[paulo]

Fix race that prevents submitting compaction for a table when executor is full

Patch by Dimitar Dimitrov; Reviewed by Paulo Motta for CASSANDRA-13801


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35e6d613
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35e6d613
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35e6d613

Branch: refs/heads/cassandra-3.0
Commit: 35e6d61361e699908d73c277da7d9ac3390f6e5d
Parents: 02aba73
Author: dimitarndimitrov 
Authored: Wed Dec 6 00:53:00 2017 +0200
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:24 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5200eb1..33fd264 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 2e69b6f..3350b20 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -111,7 +111,8 @@ public class CompactionManager implements 
CompactionManagerMBean
 private final static CompactionExecutor cacheCleanupExecutor = new 
CacheCleanupExecutor();
 
 private final CompactionMetrics metrics = new CompactionMetrics(executor, 
validationExecutor);
-private final Multiset compactingCF = 
ConcurrentHashMultiset.create();
+@VisibleForTesting
+final Multiset compactingCF = 
ConcurrentHashMultiset.create();
 
 private final RateLimiter compactionRateLimiter = 
RateLimiter.create(Double.MAX_VALUE);
 
@@ -155,6 +156,12 @@ public class CompactionManager implements 
CompactionManagerMBean
 return Collections.emptyList();
 }
 
+/**
+ * If a CF is currently being compacted, and there are no idle 
threads, submitBackground should be a no-op;
+ * we can wait for the current compaction to finish and re-submit when 
more information is available.
+ * Otherwise, we should submit at least one task to prevent starvation 
by busier CFs, and more if there
+ * are idle threads stil. (CASSANDRA-4310)
+ */
 int count = compactingCF.count(cfs);
 if (count > 0 && executor.getActiveCount() >= 
executor.getMaximumPoolSize())
 {
@@ -171,10 +178,9 @@ public class CompactionManager implements 
CompactionManagerMBean
 List futures = new ArrayList<>(1);
 Future fut = executor.submitIfRunning(new 
BackgroundCompactionCandidate(cfs), "background task");
 if (!fut.isCancelled())
-{
-compactingCF.add(cfs);
 futures.add(fut);
-}
+else
+compactingCF.remove(cfs);
 return futures;
 }
 
@@ -233,6 +239,7 @@ public class CompactionManager implements 
CompactionManagerMBean
 
 BackgroundCompactionCandidate(ColumnFamilyStore cfs)
 {
+compactingCF.add(cfs);
 this.cfs = cfs;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java 
b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
new file mode 100644
index 000..0b391a5
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with 

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[paulo]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d121691
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d121691
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d121691

Branch: refs/heads/cassandra-3.0
Commit: 0d12169167fdf58cbf619dee693a4d4e29846a7e
Parents: 4a2b516 35e6d61
Author: Paulo Motta 
Authored: Wed Dec 13 06:40:56 2017 +1100
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:56 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d121691/CHANGES.txt
--
diff --cc CHANGES.txt
index 1ca7902,33fd264..6bfddcc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,5 +1,24 @@@
 -2.2.12
 +3.0.16
 + * Extra range tombstone bound creates double rows (CASSANDRA-14008)
 + * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
 + * Accept role names containing forward-slash (CASSANDRA-14088)
 + * Optimize CRC check chance probability calculations (CASSANDRA-14094)
 + * Fix cleanup on keyspace with no replicas (CASSANDRA-13526)
 + * Fix updating base table rows with TTL not removing materialized view 
entries (CASSANDRA-14071)
 + * Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
 + * More frequent commitlog chained markers (CASSANDRA-13987)
 + * Fix serialized size of DataLimits (CASSANDRA-14057)
 + * Add flag to allow dropping oversized read repair mutations 
(CASSANDRA-13975)
 + * Fix SSTableLoader logger message (CASSANDRA-14003)
 + * Fix repair race that caused gossip to block (CASSANDRA-13849)
 + * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
 + * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
 + * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)
 + * Provide a JMX call to sync schema with local storage (CASSANDRA-13954)
 + * Mishandling of cells for removed/dropped columns when reading legacy files 
(CASSANDRA-13939)
 + * Deserialise sstable metadata in nodetool verify (CASSANDRA-13922)
 +Merged from 2.2:
+  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
   * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
   * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
  

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d121691/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[paulo]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc9bb8df
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc9bb8df
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc9bb8df

Branch: refs/heads/trunk
Commit: dc9bb8df03e0bcfeda6d3deeacf4cb750ca1d7f5
Parents: e5bb7f3 f6381db
Author: Paulo Motta 
Authored: Wed Dec 13 06:43:12 2017 +1100
Committer: Paulo Motta 
Committed: Wed Dec 13 06:44:13 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 +
 .../db/compaction/CompactionsBytemanTest.java   | 22 
 3 files changed, 34 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc9bb8df/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc9bb8df/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index cc4d078,9cd18a9..bf3b815
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -124,10 -120,10 +124,11 @@@ public class CompactionManager implemen
  private final CompactionExecutor executor = new CompactionExecutor();
  private final CompactionExecutor validationExecutor = new 
ValidationExecutor();
  private final static CompactionExecutor cacheCleanupExecutor = new 
CacheCleanupExecutor();
 +private final CompactionExecutor viewBuildExecutor = new 
ViewBuildExecutor();
  
 -private final CompactionMetrics metrics = new CompactionMetrics(executor, 
validationExecutor);
 +private final CompactionMetrics metrics = new CompactionMetrics(executor, 
validationExecutor, viewBuildExecutor);
- private final Multiset compactingCF = 
ConcurrentHashMultiset.create();
+ @VisibleForTesting
+ final Multiset compactingCF = 
ConcurrentHashMultiset.create();
  
  private final RateLimiter compactionRateLimiter = 
RateLimiter.create(Double.MAX_VALUE);
  

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc9bb8df/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
index 3ca01c1,0b391a5..2519389
--- a/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
@@@ -25,8 -23,9 +25,9 @@@ import org.junit.runner.RunWith
  
  import org.apache.cassandra.cql3.CQLTester;
  import org.apache.cassandra.db.ColumnFamilyStore;
 -import org.apache.cassandra.db.Keyspace;
+ import org.apache.cassandra.utils.FBUtilities;
  import org.jboss.byteman.contrib.bmunit.BMRule;
 +import org.jboss.byteman.contrib.bmunit.BMRules;
  import org.jboss.byteman.contrib.bmunit.BMUnitRunner;
  
  import static org.junit.Assert.assertEquals;
@@@ -34,89 -33,24 +35,110 @@@
  @RunWith(BMUnitRunner.class)
  public class CompactionsBytemanTest extends CQLTester
  {
 +/*
 +Return false for the first time hasAvailableDiskSpace is called. i.e 
first SSTable is too big
 +Create 5 SSTables. After compaction, there should be 2 left - 1 as the 9 
SStables which were merged,
 +and the other the SSTable that was 'too large' and failed the 
hasAvailableDiskSpace check
 + */
 +@Test
 +@BMRules(rules = { @BMRule(name = "One SSTable too big for remaining disk 
space test",
 +targetClass = "Directories",
 +targetMethod = "hasAvailableDiskSpace",
 +condition = "not flagged(\"done\")",
 +action = "flag(\"done\"); return false;") } )
 +public void testSSTableNotEnoughDiskSpaceForCompactionGetsDropped() 
throws Throwable
 +{
 +createLowGCGraceTable();
 +final ColumnFamilyStore cfs = getCurrentColumnFamilyStore();
 +for (int i = 0; i < 5; i++)
 +{
 +createPossiblyExpiredSSTable(cfs, false);
 +}
 +assertEquals(5, 
getCurrentColumnFamilyStore().getLiveSSTables().size());
 +cfs.forceMajorCompaction(false);
 +assertEquals(2, 
getCurrentColumnFamilyStore().getLiveSSTables().size());
 +dropTable("DROP TABLE %s");
 +}
 +
 +/*
 +Always return false for hasAvailableDiskSpace. i.e node has no more space
 +Create 2 expired SSTables and 1 long lived one. After compaction, there 
should only be 1 left,
 +as the 2 expired ones would have been compacted away.
 

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[paulo]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f6381db8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f6381db8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f6381db8

Branch: refs/heads/trunk
Commit: f6381db8edf1c532f881adb3806205cd24b0b67c
Parents: 0525145 0d12169
Author: Paulo Motta 
Authored: Wed Dec 13 06:41:11 2017 +1100
Committer: Paulo Motta 
Committed: Wed Dec 13 06:41:11 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6381db8/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6381db8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[04/10] cassandra git commit: Fix race that prevents submitting compaction for a table when executor is full

[paulo]

Fix race that prevents submitting compaction for a table when executor is full

Patch by Dimitar Dimitrov; Reviewed by Paulo Motta for CASSANDRA-13801


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35e6d613
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35e6d613
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35e6d613

Branch: refs/heads/trunk
Commit: 35e6d61361e699908d73c277da7d9ac3390f6e5d
Parents: 02aba73
Author: dimitarndimitrov 
Authored: Wed Dec 6 00:53:00 2017 +0200
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:24 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5200eb1..33fd264 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 2e69b6f..3350b20 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -111,7 +111,8 @@ public class CompactionManager implements 
CompactionManagerMBean
 private final static CompactionExecutor cacheCleanupExecutor = new 
CacheCleanupExecutor();
 
 private final CompactionMetrics metrics = new CompactionMetrics(executor, 
validationExecutor);
-private final Multiset compactingCF = 
ConcurrentHashMultiset.create();
+@VisibleForTesting
+final Multiset compactingCF = 
ConcurrentHashMultiset.create();
 
 private final RateLimiter compactionRateLimiter = 
RateLimiter.create(Double.MAX_VALUE);
 
@@ -155,6 +156,12 @@ public class CompactionManager implements 
CompactionManagerMBean
 return Collections.emptyList();
 }
 
+/**
+ * If a CF is currently being compacted, and there are no idle 
threads, submitBackground should be a no-op;
+ * we can wait for the current compaction to finish and re-submit when 
more information is available.
+ * Otherwise, we should submit at least one task to prevent starvation 
by busier CFs, and more if there
+ * are idle threads stil. (CASSANDRA-4310)
+ */
 int count = compactingCF.count(cfs);
 if (count > 0 && executor.getActiveCount() >= 
executor.getMaximumPoolSize())
 {
@@ -171,10 +178,9 @@ public class CompactionManager implements 
CompactionManagerMBean
 List futures = new ArrayList<>(1);
 Future fut = executor.submitIfRunning(new 
BackgroundCompactionCandidate(cfs), "background task");
 if (!fut.isCancelled())
-{
-compactingCF.add(cfs);
 futures.add(fut);
-}
+else
+compactingCF.remove(cfs);
 return futures;
 }
 
@@ -233,6 +239,7 @@ public class CompactionManager implements 
CompactionManagerMBean
 
 BackgroundCompactionCandidate(ColumnFamilyStore cfs)
 {
+compactingCF.add(cfs);
 this.cfs = cfs;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java 
b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
new file mode 100644
index 000..0b391a5
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the 

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[paulo]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d121691
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d121691
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d121691

Branch: refs/heads/cassandra-3.11
Commit: 0d12169167fdf58cbf619dee693a4d4e29846a7e
Parents: 4a2b516 35e6d61
Author: Paulo Motta 
Authored: Wed Dec 13 06:40:56 2017 +1100
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:56 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d121691/CHANGES.txt
--
diff --cc CHANGES.txt
index 1ca7902,33fd264..6bfddcc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,5 +1,24 @@@
 -2.2.12
 +3.0.16
 + * Extra range tombstone bound creates double rows (CASSANDRA-14008)
 + * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
 + * Accept role names containing forward-slash (CASSANDRA-14088)
 + * Optimize CRC check chance probability calculations (CASSANDRA-14094)
 + * Fix cleanup on keyspace with no replicas (CASSANDRA-13526)
 + * Fix updating base table rows with TTL not removing materialized view 
entries (CASSANDRA-14071)
 + * Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
 + * More frequent commitlog chained markers (CASSANDRA-13987)
 + * Fix serialized size of DataLimits (CASSANDRA-14057)
 + * Add flag to allow dropping oversized read repair mutations 
(CASSANDRA-13975)
 + * Fix SSTableLoader logger message (CASSANDRA-14003)
 + * Fix repair race that caused gossip to block (CASSANDRA-13849)
 + * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
 + * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
 + * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)
 + * Provide a JMX call to sync schema with local storage (CASSANDRA-13954)
 + * Mishandling of cells for removed/dropped columns when reading legacy files 
(CASSANDRA-13939)
 + * Deserialise sstable metadata in nodetool verify (CASSANDRA-13922)
 +Merged from 2.2:
+  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
   * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
   * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
  

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d121691/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[paulo]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d121691
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d121691
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d121691

Branch: refs/heads/trunk
Commit: 0d12169167fdf58cbf619dee693a4d4e29846a7e
Parents: 4a2b516 35e6d61
Author: Paulo Motta 
Authored: Wed Dec 13 06:40:56 2017 +1100
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:56 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d121691/CHANGES.txt
--
diff --cc CHANGES.txt
index 1ca7902,33fd264..6bfddcc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,5 +1,24 @@@
 -2.2.12
 +3.0.16
 + * Extra range tombstone bound creates double rows (CASSANDRA-14008)
 + * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
 + * Accept role names containing forward-slash (CASSANDRA-14088)
 + * Optimize CRC check chance probability calculations (CASSANDRA-14094)
 + * Fix cleanup on keyspace with no replicas (CASSANDRA-13526)
 + * Fix updating base table rows with TTL not removing materialized view 
entries (CASSANDRA-14071)
 + * Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
 + * More frequent commitlog chained markers (CASSANDRA-13987)
 + * Fix serialized size of DataLimits (CASSANDRA-14057)
 + * Add flag to allow dropping oversized read repair mutations 
(CASSANDRA-13975)
 + * Fix SSTableLoader logger message (CASSANDRA-14003)
 + * Fix repair race that caused gossip to block (CASSANDRA-13849)
 + * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
 + * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
 + * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)
 + * Provide a JMX call to sync schema with local storage (CASSANDRA-13954)
 + * Mishandling of cells for removed/dropped columns when reading legacy files 
(CASSANDRA-13939)
 + * Deserialise sstable metadata in nodetool verify (CASSANDRA-13922)
 +Merged from 2.2:
+  * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
   * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
   * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
  

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d121691/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[03/10] cassandra git commit: Fix race that prevents submitting compaction for a table when executor is full

[paulo]

Fix race that prevents submitting compaction for a table when executor is full

Patch by Dimitar Dimitrov; Reviewed by Paulo Motta for CASSANDRA-13801


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35e6d613
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35e6d613
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35e6d613

Branch: refs/heads/cassandra-3.11
Commit: 35e6d61361e699908d73c277da7d9ac3390f6e5d
Parents: 02aba73
Author: dimitarndimitrov 
Authored: Wed Dec 6 00:53:00 2017 +0200
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:24 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5200eb1..33fd264 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 2e69b6f..3350b20 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -111,7 +111,8 @@ public class CompactionManager implements 
CompactionManagerMBean
 private final static CompactionExecutor cacheCleanupExecutor = new 
CacheCleanupExecutor();
 
 private final CompactionMetrics metrics = new CompactionMetrics(executor, 
validationExecutor);
-private final Multiset compactingCF = 
ConcurrentHashMultiset.create();
+@VisibleForTesting
+final Multiset compactingCF = 
ConcurrentHashMultiset.create();
 
 private final RateLimiter compactionRateLimiter = 
RateLimiter.create(Double.MAX_VALUE);
 
@@ -155,6 +156,12 @@ public class CompactionManager implements 
CompactionManagerMBean
 return Collections.emptyList();
 }
 
+/**
+ * If a CF is currently being compacted, and there are no idle 
threads, submitBackground should be a no-op;
+ * we can wait for the current compaction to finish and re-submit when 
more information is available.
+ * Otherwise, we should submit at least one task to prevent starvation 
by busier CFs, and more if there
+ * are idle threads stil. (CASSANDRA-4310)
+ */
 int count = compactingCF.count(cfs);
 if (count > 0 && executor.getActiveCount() >= 
executor.getMaximumPoolSize())
 {
@@ -171,10 +178,9 @@ public class CompactionManager implements 
CompactionManagerMBean
 List futures = new ArrayList<>(1);
 Future fut = executor.submitIfRunning(new 
BackgroundCompactionCandidate(cfs), "background task");
 if (!fut.isCancelled())
-{
-compactingCF.add(cfs);
 futures.add(fut);
-}
+else
+compactingCF.remove(cfs);
 return futures;
 }
 
@@ -233,6 +239,7 @@ public class CompactionManager implements 
CompactionManagerMBean
 
 BackgroundCompactionCandidate(ColumnFamilyStore cfs)
 {
+compactingCF.add(cfs);
 this.cfs = cfs;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java 
b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
new file mode 100644
index 000..0b391a5
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with 

[01/10] cassandra git commit: Fix race that prevents submitting compaction for a table when executor is full

[paulo]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 02aba7343 -> 35e6d6136
  refs/heads/cassandra-3.0 4a2b516a3 -> 0d1216916
  refs/heads/cassandra-3.11 052514542 -> f6381db8e
  refs/heads/trunk e5bb7f343 -> dc9bb8df0


Fix race that prevents submitting compaction for a table when executor is full

Patch by Dimitar Dimitrov; Reviewed by Paulo Motta for CASSANDRA-13801


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35e6d613
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35e6d613
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35e6d613

Branch: refs/heads/cassandra-2.2
Commit: 35e6d61361e699908d73c277da7d9ac3390f6e5d
Parents: 02aba73
Author: dimitarndimitrov 
Authored: Wed Dec 6 00:53:00 2017 +0200
Committer: Paulo Motta 
Committed: Wed Dec 13 06:40:24 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5200eb1..33fd264 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Fix race that prevents submitting compaction for a table when executor is 
full (CASSANDRA-13801)
  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 2e69b6f..3350b20 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -111,7 +111,8 @@ public class CompactionManager implements 
CompactionManagerMBean
 private final static CompactionExecutor cacheCleanupExecutor = new 
CacheCleanupExecutor();
 
 private final CompactionMetrics metrics = new CompactionMetrics(executor, 
validationExecutor);
-private final Multiset compactingCF = 
ConcurrentHashMultiset.create();
+@VisibleForTesting
+final Multiset compactingCF = 
ConcurrentHashMultiset.create();
 
 private final RateLimiter compactionRateLimiter = 
RateLimiter.create(Double.MAX_VALUE);
 
@@ -155,6 +156,12 @@ public class CompactionManager implements 
CompactionManagerMBean
 return Collections.emptyList();
 }
 
+/**
+ * If a CF is currently being compacted, and there are no idle 
threads, submitBackground should be a no-op;
+ * we can wait for the current compaction to finish and re-submit when 
more information is available.
+ * Otherwise, we should submit at least one task to prevent starvation 
by busier CFs, and more if there
+ * are idle threads stil. (CASSANDRA-4310)
+ */
 int count = compactingCF.count(cfs);
 if (count > 0 && executor.getActiveCount() >= 
executor.getMaximumPoolSize())
 {
@@ -171,10 +178,9 @@ public class CompactionManager implements 
CompactionManagerMBean
 List futures = new ArrayList<>(1);
 Future fut = executor.submitIfRunning(new 
BackgroundCompactionCandidate(cfs), "background task");
 if (!fut.isCancelled())
-{
-compactingCF.add(cfs);
 futures.add(fut);
-}
+else
+compactingCF.remove(cfs);
 return futures;
 }
 
@@ -233,6 +239,7 @@ public class CompactionManager implements 
CompactionManagerMBean
 
 BackgroundCompactionCandidate(ColumnFamilyStore cfs)
 {
+compactingCF.add(cfs);
 this.cfs = cfs;
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/35e6d613/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java 
b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
new file mode 100644
index 000..0b391a5
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * 

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[paulo]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f6381db8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f6381db8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f6381db8

Branch: refs/heads/cassandra-3.11
Commit: f6381db8edf1c532f881adb3806205cd24b0b67c
Parents: 0525145 0d12169
Author: Paulo Motta 
Authored: Wed Dec 13 06:41:11 2017 +1100
Committer: Paulo Motta 
Committed: Wed Dec 13 06:41:11 2017 +1100

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java| 15 --
 .../db/compaction/CompactionsBytemanTest.java   | 56 
 3 files changed, 68 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6381db8/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6381db8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14106) utest failed: DistributionSequenceTest.setSeed() and simpleSequence()

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288102#comment-16288102
 ] 

Jeff Jirsa commented on CASSANDRA-14106:


[~bdeggleston] - you recently touched this for CASSANDRA-14090 - and this 
undoes your change. Want to weigh in?

> utest failed: DistributionSequenceTest.setSeed() and simpleSequence()
> -
>
> Key: CASSANDRA-14106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> To reproduce:
> {noformat}
> $ ant stress-test -Dtest.name=DistributionSequenceTest
> {noformat}
> {noformat}
> stress-test:
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest
> [junit] Testsuite: 
> org.apache.cassandra.stress.generate.DistributionSequenceTest Tests run: 4, 
> Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
> [junit]
> [junit] Testcase: 
> simpleSequence(org.apache.cassandra.stress.generate.DistributionSequenceTest):
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.simpleSequence(DistributionSequenceTest.java:37)
> [junit]
> [junit]
> [junit] Testcase: 
> setSeed(org.apache.cassandra.stress.generate.DistributionSequenceTest):   
> FAILED
> [junit] expected:<5> but was:<4>
> [junit] junit.framework.AssertionFailedError: expected:<5> but was:<4>
> [junit] at 
> org.apache.cassandra.stress.generate.DistributionSequenceTest.setSeed(DistributionSequenceTest.java:111)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.stress.generate.DistributionSequenceTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-6878) Test different setra values in EC2 to find the best performance

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-6878:
--
Labels:   (was: proposed-wontfix)

> Test different setra values in EC2 to find the best performance
> ---
>
> Key: CASSANDRA-6878
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6878
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Joaquin Casares
>Priority: Minor
>
> Tests should be run against:
> * Ephemeral devices
> * RAID0 ephemeral devices
> * SSD devices
> * RAID0 SSD devices
> The current recommendation is:
> {CODE}
> sudo blockdev --setra 128 /dev/
> {CODE}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-6878) Test different setra values in EC2 to find the best performance

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa resolved CASSANDRA-6878.
---
Resolution: Won't Fix

Closing as wont-fix, but if people care to share RA values here, that sounds 
great. 

> Test different setra values in EC2 to find the best performance
> ---
>
> Key: CASSANDRA-6878
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6878
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Joaquin Casares
>Priority: Minor
>
> Tests should be run against:
> * Ephemeral devices
> * RAID0 ephemeral devices
> * SSD devices
> * RAID0 SSD devices
> The current recommendation is:
> {CODE}
> sudo blockdev --setra 128 /dev/
> {CODE}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-6878) Test different setra values in EC2 to find the best performance

[Joaquin Casares (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288082#comment-16288082
 ] 

Joaquin Casares commented on CASSANDRA-6878:


Hey [~jjirsa],

I'm fine marking this as a won't fix. Perhaps in the future, people can leave 
comments on this ticket if certain setra values help them. Ideally, this 
information along with a workload and hardware description would go into the 
Cassandra documentation.

Thanks for grooming this old ticket!

> Test different setra values in EC2 to find the best performance
> ---
>
> Key: CASSANDRA-6878
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6878
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Joaquin Casares
>Priority: Minor
>  Labels: proposed-wontfix
>
> Tests should be run against:
> * Ephemeral devices
> * RAID0 ephemeral devices
> * SSD devices
> * RAID0 SSD devices
> The current recommendation is:
> {CODE}
> sudo blockdev --setra 128 /dev/
> {CODE}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14108) Improve commit log chain marker updating

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288075#comment-16288075
 ] 

Ariel Weisberg commented on CASSANDRA-14108:


I am a little surprised we did this change originally to 3.0 and 3.11 since 
it's more of an enhancement.

[Extra line 
break|https://github.com/apache/cassandra/compare/trunk...jasobrown:13987-followup-trunk?expand=1#diff-8e54554f3635ce0e8411435dd5896f4dR81].
 It's on all 3 branches.
[Another extra line 
break.https://github.com/apache/cassandra/compare/trunk...jasobrown:13987-followup-trunk?expand=1#diff-8e54554f3635ce0e8411435dd5896f4dR241]

I like the approach of quantizing. It's simple. Besides the whitespace +1.

> Improve commit log chain marker updating
> 
>
> Key: CASSANDRA-14108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> CASSANDRA-13987 addressed the commit log behavior change that was introduced 
> with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
> review and found a bug as well as having some concerns about the 
> configuration. He and I discussed offline, and agreed on some improvements. 
> Instead of requiring users to configure a deep, dark implementation detail 
> like the commit log chained markers (via {{commitlog_marker_period_in_ms}} in 
> the yaml), we decided it is best to eliminate thew configuration and always 
> update the chained markers (when in periodic mode). 
> The bug [~aweisberg] found was when the chained marker update is not a value 
> that evenly divides into the periodic sync mode value, we would not sync in 
> an expected manner. For example if the marker interval is 9 seconds, and the 
> sync interval is 10 seconds, we would update the markers at time9, but we 
> would then sleep for another 9 seconds, and when we wake up at time18, it is 
> then that we flush - 8 seconds later than we should have. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12208) Estimated droppable tombstones given by sstablemetadata counts tombstones that aren't actually "droppable"

[Daniele Rolando (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288056#comment-16288056
 ] 

Daniele Rolando commented on CASSANDRA-12208:
-

[~krummas] Hey, I see that your patch is present in the [3.11 
branch|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java],
 however when I run the sstablemetadata binary that comes with 
cassandra-tools=3.11.1 I'm not able to specify the {{--gc_grace_seconds}} 
option.

I'm using cassandra-tools_3.11.1_all.deb from 
http://dl.bintray.com/apache/cassandra/pool/main/c/cassandra/
{code}
~> sstablemetadata
Usage: sstablemetadata 

~>  sstablemetadata --gc_grace_seconds=86400 
/lb-80107-big-Data.db
No such file: --gc_grace_seconds=86400
...
{code}

Am I using it wrong or do the new binaries don't contain your patch?

> Estimated droppable tombstones given by sstablemetadata counts tombstones 
> that aren't actually "droppable"
> --
>
> Key: CASSANDRA-12208
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12208
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Thanh
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 3.0.9, 3.10
>
>
> => "Estimated droppable tombstones" given by *sstablemetadata* counts 
> tombstones that aren't actually "droppable"
> To be clear, the "Estimated droppable tombstones" calculation counts 
> tombstones that have not yet passed gc_grace_seconds as droppable tombstones, 
> which is unexpected, since such tombstones aren't droppable.
> To observe the problem:
> Create a table using the default gc_grace_seconds (default gc_grace_seconds 
> is 86400 is 1 day).
> Populate the table with a couple of records.
> Do a delete.
> Do a "nodetool flush" to flush the memtable to disk.
> Do an "sstablemetadata " to get the metadata of the sstable you just 
> created by doing the flush, and observe that the Estimated droppable 
> tombstones is greater than 0.0 (actual value depends on the total number 
> inserts/updates/deletes that you did before triggered the flush)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14110) ViewLongTest.testConflictResolution() failed

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14110:
---
Summary: ViewLongTest.testConflictResolution() failed  (was: ViewLongTest 
failed)

> ViewLongTest.testConflictResolution() failed
> 
>
> Key: CASSANDRA-14110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>
> The test works fine on my mac, but failed in the 
> [CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:
> {noformat}
> $ ant long-test -Dtest.name=ViewLongTest
> ...
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> [junit] at java.lang.Thread.run(Thread.java:745)
> [junit]
> [junit]
> [junit] Testcase: 
> testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
> ERROR
> [junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
> [junit] at 
> com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
> [junit] at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> [junit] at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
> [junit] at 
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
> [junit] at 
> org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
> [junit] at 
> org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
> [junit] Caused by: 
> com.datastax.driver.core.exceptions.OperationTimedOutException: 
> [localhost/127.0.0.1:40644] Timed out waiting for server response
> [junit] at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
> [junit] at 
> com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> [junit] at 
> com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> [junit] at 
> 

[jira] [Updated] (CASSANDRA-14110) ViewLongTest failed

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14110:
---
Description: 
The test works fine on my mac, but failed in the 
[CircleCI|https://circleci.com/gh/cooldoger/cassandra/157]:

{noformat}
$ ant long-test -Dtest.name=ViewLongTest
...
[junit] Testcase: 
testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
ERROR
[junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
[junit] at 
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
[junit] at 
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
[junit] at 
org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
[junit] at 
org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
[junit] Caused by: 
com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
[junit] at 
com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
[junit] at 
com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Testcase: 
testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
ERROR
[junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
[junit] at 
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
[junit] at 
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
[junit] at 
org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
[junit] at 
org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
[junit] Caused by: 
com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
[junit] at 
com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
[junit] at 
com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Test org.apache.cassandra.cql3.ViewLongTest FAILED
...
{noformat}

  was:
The test works fine on my mac, but failed in the [CircleCI 
title|https://circleci.com/gh/cooldoger/cassandra/157]:

{noformat}
$ ant long-test -Dtest.name=ViewLongTest
...
[junit] Testcase: 
testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
ERROR

[jira] [Created] (CASSANDRA-14110) ViewLongTest failed

[Jay Zhuang (JIRA)]

Jay Zhuang created CASSANDRA-14110:
--

 Summary: ViewLongTest failed
 Key: CASSANDRA-14110
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14110
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Jay Zhuang


The test works fine on my mac, but failed in the [CircleCI 
title|https://circleci.com/gh/cooldoger/cassandra/157]:

{noformat}
$ ant long-test -Dtest.name=ViewLongTest
...
[junit] Testcase: 
testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
ERROR
[junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
[junit] at 
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
[junit] at 
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
[junit] at 
org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
[junit] at 
org.apache.cassandra.cql3.ViewLongTest.testConflictResolution(ViewLongTest.java:138)
[junit] Caused by: 
com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
[junit] at 
com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
[junit] at 
com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Testcase: 
testConflictResolution(org.apache.cassandra.cql3.ViewLongTest):   Caused an 
ERROR
[junit] [localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44)
[junit] at 
com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26)
[junit] at 
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
[junit] at 
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
[junit] at 
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:51)
[junit] at 
org.apache.cassandra.cql3.CQLTester.executeNet(CQLTester.java:829)
[junit] at 
org.apache.cassandra.cql3.ViewLongTest.end(ViewLongTest.java:66)
[junit] Caused by: 
com.datastax.driver.core.exceptions.OperationTimedOutException: 
[localhost/127.0.0.1:40644] Timed out waiting for server response
[junit] at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:800)
[junit] at 
com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1382)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
[junit] at 
com.datastax.shaded.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
[junit] at 
com.datastax.shaded.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Test org.apache.cassandra.cql3.ViewLongTest FAILED
...
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: 

[jira] [Updated] (CASSANDRA-14073) Prevent replacement nodes from skipping bootstrapping without allow_unsafe_replace:true

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-14073:
---
Fix Version/s: (was: 3.11.2)
   3.11.x

> Prevent replacement nodes from skipping bootstrapping without 
> allow_unsafe_replace:true
> ---
>
> Key: CASSANDRA-14073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14073
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Vincent White
>Assignee: Vincent White
>Priority: Minor
>  Labels: patch
> Fix For: 3.11.x
>
>
> I've noticed that when replacing a node with replace_address it can skip 
> bootstrapping if it is listed in its own seed list. This probably shouldn't 
> be allowed without the allow_unsafe_replace option set to true as is required 
> when using auto_bootstrap: false in combination with replace_address. Patch 
> [here|https://github.com/vincewhite/cassandra/commits/replace_address_seed_list]
>  and an attempt at a dtest 
> [here|https://github.com/vincewhite/cassandra-dtest/commits/unsafe_replace]. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa resolved CASSANDRA-14008.

Resolution: Fixed

Thanks Aleksey. Committed as {{4a2b516a3488ab04ee3338e74397b8c6d69e6d43}}

> RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x
> 
>
> Key: CASSANDRA-14008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14008
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>  Labels: correctness
> Fix For: 3.0.16, 3.11.2
>
>
> In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion 
> and isn't a complex deletion to appear between two cells with the same 
> clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as 
> two distinct CQL rows, despite having the same clustering prefix.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jjirsa]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05251454
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05251454
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05251454

Branch: refs/heads/cassandra-3.11
Commit: 052514542220e66e8b755aaf8b5e015c8416b5a1
Parents: a7c45be 4a2b516
Author: Jeff Jirsa 
Authored: Tue Dec 12 09:52:50 2017 -0800
Committer: Jeff Jirsa 
Committed: Tue Dec 12 09:55:39 2017 -0800

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/db/LegacyLayout.java   | 103 ---
 ...egacy_ka_repeated_rt-ka-1-CompressionInfo.db | Bin 0 -> 43 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-Data.db | Bin 0 -> 422 bytes
 ...pace1-legacy_ka_repeated_rt-ka-1-Digest.sha1 |   1 +
 ...yspace1-legacy_ka_repeated_rt-ka-1-Filter.db | Bin 0 -> 176 bytes
 ...eyspace1-legacy_ka_repeated_rt-ka-1-Index.db | Bin 0 -> 570 bytes
 ...ce1-legacy_ka_repeated_rt-ka-1-Statistics.db | Bin 0 -> 4478 bytes
 ...space1-legacy_ka_repeated_rt-ka-1-Summary.db | Bin 0 -> 92 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-TOC.txt |   8 ++
 .../apache/cassandra/db/LegacyLayoutTest.java   | 100 ++
 11 files changed, 176 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/05251454/CHANGES.txt
--
diff --cc CHANGES.txt
index 116c7e9,1ca7902..da40341
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 -3.0.16
 +3.11.2
 + * Fix imbalanced disks when replacing node with same address with JBOD 
(CASSANDRA-14084)
 + * Reload compaction strategies when disk boundaries are invalidated 
(CASSANDRA-13948)
 + * Remove OpenJDK log warning (CASSANDRA-13916)
 + * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079)
 + * Cache disk boundaries (CASSANDRA-13215)
 + * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
 + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
 + * Update jackson JSON jars (CASSANDRA-13949)
 + * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)
 +Merged from 3.0:
+  * Extra range tombstone bound creates double rows (CASSANDRA-14008)
   * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
   * Accept role names containing forward-slash (CASSANDRA-14088)
   * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/05251454/src/java/org/apache/cassandra/db/LegacyLayout.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/05251454/test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
--
diff --cc test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
index 6103727,715d7c9..be695f2
--- a/test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
+++ b/test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
@@@ -19,12 -19,19 +19,22 @@@
  package org.apache.cassandra.db;
  
  import java.nio.ByteBuffer;
+ import java.nio.file.Files;
+ import java.nio.file.Path;
+ import java.nio.file.Paths;
  
++import org.junit.AfterClass;
++import org.junit.BeforeClass;
  import org.junit.Test;
  
+ import org.apache.cassandra.SchemaLoader;
++import org.apache.cassandra.Util;
  import org.apache.cassandra.config.CFMetaData;
  import org.apache.cassandra.config.ColumnDefinition;
+ import org.apache.cassandra.config.DatabaseDescriptor;
  import org.apache.cassandra.cql3.ColumnIdentifier;
+ import org.apache.cassandra.cql3.QueryProcessor;
+ import org.apache.cassandra.cql3.UntypedResultSet;
  import org.apache.cassandra.db.marshal.Int32Type;
  import org.apache.cassandra.db.marshal.SetType;
  import org.apache.cassandra.db.partitions.PartitionUpdate;
@@@ -37,6 -45,6 +48,24 @@@ import static org.junit.Assert.*
  
  public class LegacyLayoutTest
  {
++static Util.PartitionerSwitcher sw;
++static String KEYSPACE = "Keyspace1";
++
++@BeforeClass
++public static void setupPartitioner()
++{
++DatabaseDescriptor.daemonInitialization();
++sw = Util.switchPartitioner(Murmur3Partitioner.instance);
++SchemaLoader.loadSchema();
++SchemaLoader.createKeyspace(KEYSPACE, KeyspaceParams.simple(1));
++}
++
++@AfterClass
++public static void resetPartitioner()
++{
++sw.close();
++}
++
  @Test
  public void testFromUnfilteredRowIterator() throws Throwable
  {
@@@ -70,4 -77,78 +99,75 @@@
  assertEquals("b", l.starts[1].collectionName.name.toString());

[3/6] cassandra git commit: Extra range tombstone bound creates double rows in 3.0

[jjirsa]

Extra range tombstone bound creates double rows in 3.0

Patch by Jeff Jirsa; Reviewed by Aleksey Yeschenko for CASSANDRA-14008


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a2b516a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a2b516a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a2b516a

Branch: refs/heads/trunk
Commit: 4a2b516a3488ab04ee3338e74397b8c6d69e6d43
Parents: dd187d1
Author: Jeff Jirsa 
Authored: Sat Nov 11 09:10:22 2017 -0800
Committer: Jeff Jirsa 
Committed: Tue Dec 12 09:52:19 2017 -0800

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/db/LegacyLayout.java   | 103 ---
 ...egacy_ka_repeated_rt-ka-1-CompressionInfo.db | Bin 0 -> 43 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-Data.db | Bin 0 -> 422 bytes
 ...pace1-legacy_ka_repeated_rt-ka-1-Digest.sha1 |   1 +
 ...yspace1-legacy_ka_repeated_rt-ka-1-Filter.db | Bin 0 -> 176 bytes
 ...eyspace1-legacy_ka_repeated_rt-ka-1-Index.db | Bin 0 -> 570 bytes
 ...ce1-legacy_ka_repeated_rt-ka-1-Statistics.db | Bin 0 -> 4478 bytes
 ...space1-legacy_ka_repeated_rt-ka-1-Summary.db | Bin 0 -> 92 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-TOC.txt |   8 ++
 .../apache/cassandra/db/LegacyLayoutTest.java   |  83 +++
 11 files changed, 159 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a2b516a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20ccc4b..1ca7902 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.16
+ * Extra range tombstone bound creates double rows (CASSANDRA-14008)
  * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
  * Accept role names containing forward-slash (CASSANDRA-14088)
  * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a2b516a/src/java/org/apache/cassandra/db/LegacyLayout.java
--
diff --git a/src/java/org/apache/cassandra/db/LegacyLayout.java 
b/src/java/org/apache/cassandra/db/LegacyLayout.java
index 3ba96a6..2117dd6 100644
--- a/src/java/org/apache/cassandra/db/LegacyLayout.java
+++ b/src/java/org/apache/cassandra/db/LegacyLayout.java
@@ -1273,56 +1273,85 @@ public abstract class LegacyLayout
 return true;
 }
 
-public boolean addRangeTombstone(LegacyRangeTombstone tombstone)
+private boolean addRangeTombstone(LegacyRangeTombstone tombstone)
 {
 if (tombstone.isRowDeletion(metadata))
+return addRowTombstone(tombstone);
+else if (tombstone.isCollectionTombstone())
+return addCollectionTombstone(tombstone);
+else
+return addGenericRangeTombstone(tombstone);
+}
+
+private boolean addRowTombstone(LegacyRangeTombstone tombstone)
+{
+if (clustering != null)
 {
-if (clustering != null)
+// If we're already in the row, there might be a chance that 
there were two range tombstones
+// written, as 2.x storage format does not guarantee just one 
range tombstone, unlike 3.x.
+// We have to make sure that clustering matches, which would 
mean that tombstone is for the
+// same row.
+if (rowDeletion != null && 
clustering.equals(tombstone.start.getAsClustering(metadata)))
 {
-// If we're already in the row, there might be a chance 
that there were two range tombstones
-// written, as 2.x storage format does not guarantee just 
one range tombstone, unlike 3.x.
-// We have to make sure that clustering matches, which 
would mean that tombstone is for the
-// same row.
-if (rowDeletion != null && 
clustering.equals(tombstone.start.getAsClustering(metadata)))
+// If the tombstone superceeds the previous delete, we 
discard the previous one
+if 
(tombstone.deletionTime.supersedes(rowDeletion.deletionTime))
 {
-// If the tombstone superceeds the previous delete, we 
discard the previous one
-if 
(tombstone.deletionTime.supersedes(rowDeletion.deletionTime))
-{
-
builder.addRowDeletion(Row.Deletion.regular(tombstone.deletionTime));
-rowDeletion = tombstone;
-}
-return true;
+

[1/6] cassandra git commit: Extra range tombstone bound creates double rows in 3.0

[jjirsa]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 dd187d105 -> 4a2b516a3
  refs/heads/cassandra-3.11 a7c45be93 -> 052514542
  refs/heads/trunk 17e602b7d -> e5bb7f343


Extra range tombstone bound creates double rows in 3.0

Patch by Jeff Jirsa; Reviewed by Aleksey Yeschenko for CASSANDRA-14008


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a2b516a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a2b516a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a2b516a

Branch: refs/heads/cassandra-3.0
Commit: 4a2b516a3488ab04ee3338e74397b8c6d69e6d43
Parents: dd187d1
Author: Jeff Jirsa 
Authored: Sat Nov 11 09:10:22 2017 -0800
Committer: Jeff Jirsa 
Committed: Tue Dec 12 09:52:19 2017 -0800

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/db/LegacyLayout.java   | 103 ---
 ...egacy_ka_repeated_rt-ka-1-CompressionInfo.db | Bin 0 -> 43 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-Data.db | Bin 0 -> 422 bytes
 ...pace1-legacy_ka_repeated_rt-ka-1-Digest.sha1 |   1 +
 ...yspace1-legacy_ka_repeated_rt-ka-1-Filter.db | Bin 0 -> 176 bytes
 ...eyspace1-legacy_ka_repeated_rt-ka-1-Index.db | Bin 0 -> 570 bytes
 ...ce1-legacy_ka_repeated_rt-ka-1-Statistics.db | Bin 0 -> 4478 bytes
 ...space1-legacy_ka_repeated_rt-ka-1-Summary.db | Bin 0 -> 92 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-TOC.txt |   8 ++
 .../apache/cassandra/db/LegacyLayoutTest.java   |  83 +++
 11 files changed, 159 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a2b516a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20ccc4b..1ca7902 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.16
+ * Extra range tombstone bound creates double rows (CASSANDRA-14008)
  * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
  * Accept role names containing forward-slash (CASSANDRA-14088)
  * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a2b516a/src/java/org/apache/cassandra/db/LegacyLayout.java
--
diff --git a/src/java/org/apache/cassandra/db/LegacyLayout.java 
b/src/java/org/apache/cassandra/db/LegacyLayout.java
index 3ba96a6..2117dd6 100644
--- a/src/java/org/apache/cassandra/db/LegacyLayout.java
+++ b/src/java/org/apache/cassandra/db/LegacyLayout.java
@@ -1273,56 +1273,85 @@ public abstract class LegacyLayout
 return true;
 }
 
-public boolean addRangeTombstone(LegacyRangeTombstone tombstone)
+private boolean addRangeTombstone(LegacyRangeTombstone tombstone)
 {
 if (tombstone.isRowDeletion(metadata))
+return addRowTombstone(tombstone);
+else if (tombstone.isCollectionTombstone())
+return addCollectionTombstone(tombstone);
+else
+return addGenericRangeTombstone(tombstone);
+}
+
+private boolean addRowTombstone(LegacyRangeTombstone tombstone)
+{
+if (clustering != null)
 {
-if (clustering != null)
+// If we're already in the row, there might be a chance that 
there were two range tombstones
+// written, as 2.x storage format does not guarantee just one 
range tombstone, unlike 3.x.
+// We have to make sure that clustering matches, which would 
mean that tombstone is for the
+// same row.
+if (rowDeletion != null && 
clustering.equals(tombstone.start.getAsClustering(metadata)))
 {
-// If we're already in the row, there might be a chance 
that there were two range tombstones
-// written, as 2.x storage format does not guarantee just 
one range tombstone, unlike 3.x.
-// We have to make sure that clustering matches, which 
would mean that tombstone is for the
-// same row.
-if (rowDeletion != null && 
clustering.equals(tombstone.start.getAsClustering(metadata)))
+// If the tombstone superceeds the previous delete, we 
discard the previous one
+if 
(tombstone.deletionTime.supersedes(rowDeletion.deletionTime))
 {
-// If the tombstone superceeds the previous delete, we 
discard the previous one
-if 
(tombstone.deletionTime.supersedes(rowDeletion.deletionTime))
-{
-

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jjirsa]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05251454
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05251454
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05251454

Branch: refs/heads/trunk
Commit: 052514542220e66e8b755aaf8b5e015c8416b5a1
Parents: a7c45be 4a2b516
Author: Jeff Jirsa 
Authored: Tue Dec 12 09:52:50 2017 -0800
Committer: Jeff Jirsa 
Committed: Tue Dec 12 09:55:39 2017 -0800

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/db/LegacyLayout.java   | 103 ---
 ...egacy_ka_repeated_rt-ka-1-CompressionInfo.db | Bin 0 -> 43 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-Data.db | Bin 0 -> 422 bytes
 ...pace1-legacy_ka_repeated_rt-ka-1-Digest.sha1 |   1 +
 ...yspace1-legacy_ka_repeated_rt-ka-1-Filter.db | Bin 0 -> 176 bytes
 ...eyspace1-legacy_ka_repeated_rt-ka-1-Index.db | Bin 0 -> 570 bytes
 ...ce1-legacy_ka_repeated_rt-ka-1-Statistics.db | Bin 0 -> 4478 bytes
 ...space1-legacy_ka_repeated_rt-ka-1-Summary.db | Bin 0 -> 92 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-TOC.txt |   8 ++
 .../apache/cassandra/db/LegacyLayoutTest.java   | 100 ++
 11 files changed, 176 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/05251454/CHANGES.txt
--
diff --cc CHANGES.txt
index 116c7e9,1ca7902..da40341
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 -3.0.16
 +3.11.2
 + * Fix imbalanced disks when replacing node with same address with JBOD 
(CASSANDRA-14084)
 + * Reload compaction strategies when disk boundaries are invalidated 
(CASSANDRA-13948)
 + * Remove OpenJDK log warning (CASSANDRA-13916)
 + * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079)
 + * Cache disk boundaries (CASSANDRA-13215)
 + * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
 + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
 + * Update jackson JSON jars (CASSANDRA-13949)
 + * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)
 +Merged from 3.0:
+  * Extra range tombstone bound creates double rows (CASSANDRA-14008)
   * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
   * Accept role names containing forward-slash (CASSANDRA-14088)
   * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/05251454/src/java/org/apache/cassandra/db/LegacyLayout.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/05251454/test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
--
diff --cc test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
index 6103727,715d7c9..be695f2
--- a/test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
+++ b/test/unit/org/apache/cassandra/db/LegacyLayoutTest.java
@@@ -19,12 -19,19 +19,22 @@@
  package org.apache.cassandra.db;
  
  import java.nio.ByteBuffer;
+ import java.nio.file.Files;
+ import java.nio.file.Path;
+ import java.nio.file.Paths;
  
++import org.junit.AfterClass;
++import org.junit.BeforeClass;
  import org.junit.Test;
  
+ import org.apache.cassandra.SchemaLoader;
++import org.apache.cassandra.Util;
  import org.apache.cassandra.config.CFMetaData;
  import org.apache.cassandra.config.ColumnDefinition;
+ import org.apache.cassandra.config.DatabaseDescriptor;
  import org.apache.cassandra.cql3.ColumnIdentifier;
+ import org.apache.cassandra.cql3.QueryProcessor;
+ import org.apache.cassandra.cql3.UntypedResultSet;
  import org.apache.cassandra.db.marshal.Int32Type;
  import org.apache.cassandra.db.marshal.SetType;
  import org.apache.cassandra.db.partitions.PartitionUpdate;
@@@ -37,6 -45,6 +48,24 @@@ import static org.junit.Assert.*
  
  public class LegacyLayoutTest
  {
++static Util.PartitionerSwitcher sw;
++static String KEYSPACE = "Keyspace1";
++
++@BeforeClass
++public static void setupPartitioner()
++{
++DatabaseDescriptor.daemonInitialization();
++sw = Util.switchPartitioner(Murmur3Partitioner.instance);
++SchemaLoader.loadSchema();
++SchemaLoader.createKeyspace(KEYSPACE, KeyspaceParams.simple(1));
++}
++
++@AfterClass
++public static void resetPartitioner()
++{
++sw.close();
++}
++
  @Test
  public void testFromUnfilteredRowIterator() throws Throwable
  {
@@@ -70,4 -77,78 +99,75 @@@
  assertEquals("b", l.starts[1].collectionName.name.toString());
 

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jjirsa]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5bb7f34
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5bb7f34
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5bb7f34

Branch: refs/heads/trunk
Commit: e5bb7f343dbd6bfa476bb7c8cbfe531a3615ca49
Parents: 17e602b 0525145
Author: Jeff Jirsa 
Authored: Tue Dec 12 09:55:55 2017 -0800
Committer: Jeff Jirsa 
Committed: Tue Dec 12 09:58:06 2017 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5bb7f34/CHANGES.txt
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Extra range tombstone bound creates double rows in 3.0

[jjirsa]

Extra range tombstone bound creates double rows in 3.0

Patch by Jeff Jirsa; Reviewed by Aleksey Yeschenko for CASSANDRA-14008


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a2b516a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a2b516a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a2b516a

Branch: refs/heads/cassandra-3.11
Commit: 4a2b516a3488ab04ee3338e74397b8c6d69e6d43
Parents: dd187d1
Author: Jeff Jirsa 
Authored: Sat Nov 11 09:10:22 2017 -0800
Committer: Jeff Jirsa 
Committed: Tue Dec 12 09:52:19 2017 -0800

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/db/LegacyLayout.java   | 103 ---
 ...egacy_ka_repeated_rt-ka-1-CompressionInfo.db | Bin 0 -> 43 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-Data.db | Bin 0 -> 422 bytes
 ...pace1-legacy_ka_repeated_rt-ka-1-Digest.sha1 |   1 +
 ...yspace1-legacy_ka_repeated_rt-ka-1-Filter.db | Bin 0 -> 176 bytes
 ...eyspace1-legacy_ka_repeated_rt-ka-1-Index.db | Bin 0 -> 570 bytes
 ...ce1-legacy_ka_repeated_rt-ka-1-Statistics.db | Bin 0 -> 4478 bytes
 ...space1-legacy_ka_repeated_rt-ka-1-Summary.db | Bin 0 -> 92 bytes
 ...Keyspace1-legacy_ka_repeated_rt-ka-1-TOC.txt |   8 ++
 .../apache/cassandra/db/LegacyLayoutTest.java   |  83 +++
 11 files changed, 159 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a2b516a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20ccc4b..1ca7902 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.16
+ * Extra range tombstone bound creates double rows (CASSANDRA-14008)
  * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
  * Accept role names containing forward-slash (CASSANDRA-14088)
  * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a2b516a/src/java/org/apache/cassandra/db/LegacyLayout.java
--
diff --git a/src/java/org/apache/cassandra/db/LegacyLayout.java 
b/src/java/org/apache/cassandra/db/LegacyLayout.java
index 3ba96a6..2117dd6 100644
--- a/src/java/org/apache/cassandra/db/LegacyLayout.java
+++ b/src/java/org/apache/cassandra/db/LegacyLayout.java
@@ -1273,56 +1273,85 @@ public abstract class LegacyLayout
 return true;
 }
 
-public boolean addRangeTombstone(LegacyRangeTombstone tombstone)
+private boolean addRangeTombstone(LegacyRangeTombstone tombstone)
 {
 if (tombstone.isRowDeletion(metadata))
+return addRowTombstone(tombstone);
+else if (tombstone.isCollectionTombstone())
+return addCollectionTombstone(tombstone);
+else
+return addGenericRangeTombstone(tombstone);
+}
+
+private boolean addRowTombstone(LegacyRangeTombstone tombstone)
+{
+if (clustering != null)
 {
-if (clustering != null)
+// If we're already in the row, there might be a chance that 
there were two range tombstones
+// written, as 2.x storage format does not guarantee just one 
range tombstone, unlike 3.x.
+// We have to make sure that clustering matches, which would 
mean that tombstone is for the
+// same row.
+if (rowDeletion != null && 
clustering.equals(tombstone.start.getAsClustering(metadata)))
 {
-// If we're already in the row, there might be a chance 
that there were two range tombstones
-// written, as 2.x storage format does not guarantee just 
one range tombstone, unlike 3.x.
-// We have to make sure that clustering matches, which 
would mean that tombstone is for the
-// same row.
-if (rowDeletion != null && 
clustering.equals(tombstone.start.getAsClustering(metadata)))
+// If the tombstone superceeds the previous delete, we 
discard the previous one
+if 
(tombstone.deletionTime.supersedes(rowDeletion.deletionTime))
 {
-// If the tombstone superceeds the previous delete, we 
discard the previous one
-if 
(tombstone.deletionTime.supersedes(rowDeletion.deletionTime))
-{
-
builder.addRowDeletion(Row.Deletion.regular(tombstone.deletionTime));
-rowDeletion = tombstone;
-}
-return true;

[jira] [Updated] (CASSANDRA-14109) Prevent continuous schema exchange between 3.0 and 3.11 nodes

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-14109:
-
Status: Patch Available  (was: Open)

> Prevent continuous schema exchange between 3.0 and 3.11 nodes
> -
>
> Key: CASSANDRA-14109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14109
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Distributed Metadata
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Blocker
> Fix For: 3.11.2
>
>
> Continuous schema migrations can happen during an upgrade from 3.0.x to 3.x 
> even with versions having the patches for CASSANDRA-13441 and CASSANDRA-13559.
> The root cause is the {{cdc}} column, which is included in schema version 
> calculation in {{RowIterators.digest()}} via 
> {{SchemaKeyspace.calculateSchemaDigest()}}.
> It is possible to make the schema-version calculation between 3.0 and 3.11 
> compatible. The idea here is: 3.11 accepts both 3.0 compatible and 3.11 
> "native" schema versions. As long as there is one 3.0 node in the cluster, 
> 3.11 announces a 3.0 compatible schema version (without the {{cdc}} column). 
> When there are no (more) 3.0 nodes in the cluster, announce the "real" 3.11 
> schema version (including the {{cdc}} column). "Announce" means announcing 
> via Gossip and storing in {{system.local}}.
> The change itself is against 3.11 only. A couple of log messages have been 
> improved and some code regarding schema version checks has been moved into 
> the {{Schema}} class. Those "side changes" are carried to trunk. Because of 
> that, the 3.11 and trunk branches are different. The "real" change is in the 
> 3.11 branch.
> {{NEWS.txt}} for 3.11(only) contains upgrade notes.
> ||OSS 
> 3.11|[branch|https://github.com/apache/cassandra/compare/cassandra-3.11...snazy:schema-migration-upgrade-bug-3.11?expand=1]
> ||OSS 
> trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:schema-migration-upgrade-bug-trunk?expand=1]
> ||OSS 
> dtest|[branch|https://github.com/riptano/cassandra-dtest/compare/master...snazy:schema-migration-upgrade-bug?expand=1]
> We've verified the functionality of the patch by usual CI tests and extensive 
> tests using the new upgrade dtest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14109) Prevent continuous schema exchange between 3.0 and 3.11 nodes

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-14109:


 Summary: Prevent continuous schema exchange between 3.0 and 3.11 
nodes
 Key: CASSANDRA-14109
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14109
 Project: Cassandra
  Issue Type: Bug
  Components: Coordination, Distributed Metadata
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Blocker
 Fix For: 3.11.2


Continuous schema migrations can happen during an upgrade from 3.0.x to 3.x 
even with versions having the patches for CASSANDRA-13441 and CASSANDRA-13559.

The root cause is the {{cdc}} column, which is included in schema version 
calculation in {{RowIterators.digest()}} via 
{{SchemaKeyspace.calculateSchemaDigest()}}.

It is possible to make the schema-version calculation between 3.0 and 3.11 
compatible. The idea here is: 3.11 accepts both 3.0 compatible and 3.11 
"native" schema versions. As long as there is one 3.0 node in the cluster, 3.11 
announces a 3.0 compatible schema version (without the {{cdc}} column). When 
there are no (more) 3.0 nodes in the cluster, announce the "real" 3.11 schema 
version (including the {{cdc}} column). "Announce" means announcing via Gossip 
and storing in {{system.local}}.

The change itself is against 3.11 only. A couple of log messages have been 
improved and some code regarding schema version checks has been moved into the 
{{Schema}} class. Those "side changes" are carried to trunk. Because of that, 
the 3.11 and trunk branches are different. The "real" change is in the 3.11 
branch.

{{NEWS.txt}} for 3.11(only) contains upgrade notes.

||OSS 
3.11|[branch|https://github.com/apache/cassandra/compare/cassandra-3.11...snazy:schema-migration-upgrade-bug-3.11?expand=1]
||OSS 
trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:schema-migration-upgrade-bug-trunk?expand=1]
||OSS 
dtest|[branch|https://github.com/riptano/cassandra-dtest/compare/master...snazy:schema-migration-upgrade-bug?expand=1]

We've verified the functionality of the patch by usual CI tests and extensive 
tests using the new upgrade dtest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12978:
---
Labels:   (was: proposed-wontfix)

> mx4j -> HTTP 500 -> ConcurrentModificationException
> ---
>
> Key: CASSANDRA-12978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 
> 16GB, RAID1 SSD Commit log, RAID10 15k HDD data
>Reporter: Rob Emery
>Priority: Critical
> Fix For: 2.1.6
>
>
> We run some checks from our Monitoring software that rely on mx4j.
> The checks typically grab some xml via HTTP request and parse it. For 
> example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
> using:
> http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily
> The checks run each minute. Periodically they result in a "HTTP 500 internal 
> server error". The HTML body returned is empty.
> Experimentally we ran Cassandra in the foreground on one node and reproduced 
> the problem. this elicited the following stack trace:
> javax.management.RuntimeMBeanException: 
> java.util.ConcurrentModificationException
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
> at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
> at 
> mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
> Caused by: java.util.ConcurrentModificationException
> at 
> java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
> at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
> at 
> org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
> at 
> org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at 
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at 
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> ... 4 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa resolved CASSANDRA-12978.

Resolution: Won't Fix

> mx4j -> HTTP 500 -> ConcurrentModificationException
> ---
>
> Key: CASSANDRA-12978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 
> 16GB, RAID1 SSD Commit log, RAID10 15k HDD data
>Reporter: Rob Emery
>Priority: Critical
> Fix For: 2.1.6
>
>
> We run some checks from our Monitoring software that rely on mx4j.
> The checks typically grab some xml via HTTP request and parse it. For 
> example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
> using:
> http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily
> The checks run each minute. Periodically they result in a "HTTP 500 internal 
> server error". The HTML body returned is empty.
> Experimentally we ran Cassandra in the foreground on one node and reproduced 
> the problem. this elicited the following stack trace:
> javax.management.RuntimeMBeanException: 
> java.util.ConcurrentModificationException
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
> at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
> at 
> mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
> Caused by: java.util.ConcurrentModificationException
> at 
> java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
> at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
> at 
> org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
> at 
> org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at 
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at 
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> ... 4 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14108) Improve commit log chain marker updating

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14108:

Fix Version/s: 3.11.x
   3.0.x

> Improve commit log chain marker updating
> 
>
> Key: CASSANDRA-14108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> CASSANDRA-13987 addressed the commit log behavior change that was introduced 
> with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
> review and found a bug as well as having some concerns about the 
> configuration. He and I discussed offline, and agreed on some improvements. 
> Instead of requiring users to configure a deep, dark implementation detail 
> like the commit log chained markers (via {{commitlog_marker_period_in_ms}} in 
> the yaml), we decided it is best to eliminate thew configuration and always 
> update the chained markers (when in periodic mode). 
> The bug [~aweisberg] found was when the chained marker update is not a value 
> that evenly divides into the periodic sync mode value, we would not sync in 
> an expected manner. For example if the marker interval is 9 seconds, and the 
> sync interval is 10 seconds, we would update the markers at time9, but we 
> would then sleep for another 9 seconds, and when we wake up at time18, it is 
> then that we flush - 8 seconds later than we should have. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14108) Improve commit log chain marker updating

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287857#comment-16287857
 ] 

Jason Brown edited comment on CASSANDRA-14108 at 12/12/17 4:44 PM:
---

I've assigned [~aweisberg] as reviewer, but it'd be fantastic if [~beobal] 
could take a look, as well.


was (Author: jasobrown):
I've assigned [~aweisberg] as reviewer, but I'd be fantastic if [~beobal] could 
take a look, as well.

> Improve commit log chain marker updating
> 
>
> Key: CASSANDRA-14108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.x
>
>
> CASSANDRA-13987 addressed the commit log behavior change that was introduced 
> with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
> review and found a bug as well as having some concerns about the 
> configuration. He and I discussed offline, and agreed on some improvements. 
> Instead of requiring users to configure a deep, dark implementation detail 
> like the commit log chained markers (via {{commitlog_marker_period_in_ms}} in 
> the yaml), we decided it is best to eliminate thew configuration and always 
> update the chained markers (when in periodic mode). 
> The bug [~aweisberg] found was when the chained marker update is not a value 
> that evenly divides into the periodic sync mode value, we would not sync in 
> an expected manner. For example if the marker interval is 9 seconds, and the 
> sync interval is 10 seconds, we would update the markers at time9, but we 
> would then sleep for another 9 seconds, and when we wake up at time18, it is 
> then that we flush - 8 seconds later than we should have. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14108) Improve commit log chain marker updating

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287857#comment-16287857
 ] 

Jason Brown commented on CASSANDRA-14108:
-

I've assigned [~aweisberg] as reviewer, but I'd be fantastic if [~beobal] could 
take a look, as well.

> Improve commit log chain marker updating
> 
>
> Key: CASSANDRA-14108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.x
>
>
> CASSANDRA-13987 addressed the commit log behavior change that was introduced 
> with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
> review and found a bug as well as having some concerns about the 
> configuration. He and I discussed offline, and agreed on some improvements. 
> Instead of requiring users to configure a deep, dark implementation detail 
> like the commit log chained markers (via {{commitlog_marker_period_in_ms}} in 
> the yaml), we decided it is best to eliminate thew configuration and always 
> update the chained markers (when in periodic mode). 
> The bug [~aweisberg] found was when the chained marker update is not a value 
> that evenly divides into the periodic sync mode value, we would not sync in 
> an expected manner. For example if the marker interval is 9 seconds, and the 
> sync interval is 10 seconds, we would update the markers at time9, but we 
> would then sleep for another 9 seconds, and when we wake up at time18, it is 
> then that we flush - 8 seconds later than we should have. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14108) Improve commit log chain marker updating

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287856#comment-16287856
 ] 

Jason Brown commented on CASSANDRA-14108:
-

Branches and tests here:

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13987-followup-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13987-followup-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13987-followup-trunk]|
|[utests  
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/13987-followup-3.0]|[utests
  
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/13987-followup-3.11]|[utests
  
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/13987-followup-trunk]|
||

I've removed the confusing (and confusingly described) yaml property for 
setting the {{commitlog_marker_period_in_ms}}. Instead, I've hardcoded the 
marker interval to 100ms and it is always applied when a) using periodic mode, 
and b) not using compression or encryption.

I've fixed the bug that @ariel found by quantizing the 
{{commitlog_sync_period_in_ms}} to a multiple of the marker interval. At the 
worst, this would change the {{commitlog_sync_period_in_ms}} by up to 50 ms. I 
don't think anyone will really notice those milliseconds at a practical level, 
and if they do, they should be using batch or group mode.

I've also refactored {{AbstractCommitLogService}} so we can actually unit test 
it. The anonymous {{Runnable}} class that had 90% of the functionality of  
{{AbstractCommitLogService}} is now a full-fledged, named, and accessible 
nested class, {{AbstractCommitLogService.SyncRunnable}}. To avoid being bound 
by actual wall clock timings for correctness testing, 
{{AbstractCommitLogService.SyncRunnable}} is using the {{Clock}} abstraction; 
this makes testing sane, safe, and repeatable. {{Clock}} was introduced in 
CASSANDRA-12016, and committed to 3.10. I've backported it to 3.0, but I feel 
it's quite safe and doesn't affect anything outside of itself.


> Improve commit log chain marker updating
> 
>
> Key: CASSANDRA-14108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.x
>
>
> CASSANDRA-13987 addressed the commit log behavior change that was introduced 
> with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
> review and found a bug as well as having some concerns about the 
> configuration. He and I discussed offline, and agreed on some improvements. 
> Instead of requiring users to configure a deep, dark implementation detail 
> like the commit log chained markers (via {{commitlog_marker_period_in_ms}} in 
> the yaml), we decided it is best to eliminate thew configuration and always 
> update the chained markers (when in periodic mode). 
> The bug [~aweisberg] found was when the chained marker update is not a value 
> that evenly divides into the periodic sync mode value, we would not sync in 
> an expected manner. For example if the marker interval is 9 seconds, and the 
> sync interval is 10 seconds, we would update the markers at time9, but we 
> would then sleep for another 9 seconds, and when we wake up at time18, it is 
> then that we flush - 8 seconds later than we should have. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14108) Improve commit log chain marker updating

[Jason Brown (JIRA)]

Jason Brown created CASSANDRA-14108:
---

 Summary: Improve commit log chain marker updating
 Key: CASSANDRA-14108
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14108
 Project: Cassandra
  Issue Type: Bug
Reporter: Jason Brown
Assignee: Jason Brown
 Fix For: 4.x


CASSANDRA-13987 addressed the commit log behavior change that was introduced 
with CASSANDRA-3578. After that patch was committed, [~aweisberg] did his own 
review and found a bug as well as having some concerns about the configuration. 
He and I discussed offline, and agreed on some improvements. 

Instead of requiring users to configure a deep, dark implementation detail like 
the commit log chained markers (via {{commitlog_marker_period_in_ms}} in the 
yaml), we decided it is best to eliminate thew configuration and always update 
the chained markers (when in periodic mode). 

The bug [~aweisberg] found was when the chained marker update is not a value 
that evenly divides into the periodic sync mode value, we would not sync in an 
expected manner. For example if the marker interval is 9 seconds, and the sync 
interval is 10 seconds, we would update the markers at time9, but we would then 
sleep for another 9 seconds, and when we wake up at time18, it is then that we 
flush - 8 seconds later than we should have. 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

[Edward Ribeiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287737#comment-16287737
 ] 

Edward Ribeiro commented on CASSANDRA-12978:


Yes, sounds good to close it. 

Thanks

> mx4j -> HTTP 500 -> ConcurrentModificationException
> ---
>
> Key: CASSANDRA-12978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 
> 16GB, RAID1 SSD Commit log, RAID10 15k HDD data
>Reporter: Rob Emery
>Priority: Critical
>  Labels: proposed-wontfix
> Fix For: 2.1.6
>
>
> We run some checks from our Monitoring software that rely on mx4j.
> The checks typically grab some xml via HTTP request and parse it. For 
> example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
> using:
> http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily
> The checks run each minute. Periodically they result in a "HTTP 500 internal 
> server error". The HTML body returned is empty.
> Experimentally we ran Cassandra in the foreground on one node and reproduced 
> the problem. this elicited the following stack trace:
> javax.management.RuntimeMBeanException: 
> java.util.ConcurrentModificationException
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
> at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
> at 
> mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
> Caused by: java.util.ConcurrentModificationException
> at 
> java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
> at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
> at 
> org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
> at 
> org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at 
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at 
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> ... 4 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14107) Introduce simple key alias versioning scheme for TDE

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-14107:
---
Labels: encryption  (was: )

> Introduce simple key alias versioning scheme for TDE
> 
>
> Key: CASSANDRA-14107
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14107
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Stefan Podkowinski
>Priority: Minor
>  Labels: encryption
> Fix For: 4.x
>
>
> Handling of encryption keys as introduced in CASSANDRA-9945 takes place by 
> referencing a key alias in either cassandra.yaml, or the header of the 
> (commitlog/hints) file that has been encrypted. Using the alias as literal 
> value will work, but requires some attention when rotating keys.
> Currently each time a key is rotated (i.e. adding a new key to the keystore 
> while preserving the previous version), the alias in cassandra.yaml has to be 
> update as well and the node needs to be restarted. It would be more 
> convenient to use a symbolic reference instead. My suggestion here would be 
> to use ":last" for referring to the latest version. Omitting the 
> ":" part altogether would have the same effect. In this case 
> Cassandra always picks the latest key when the key cache has been expired.
> The non-trivial part of this suggestion is how the "latest" key is referenced 
> in the file header. If we use "latest", e.g. for the commit log header, and 
> the key gets rotated, we'd now try do decrypt the file with the new key, 
> instead of the key it has been created with. Therefor we'd have to introduce 
> an extra step that will resolve the canonical version for "latest" and refer 
> to that one during any encrypt operation. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14107) Introduce simple key alias versioning scheme for TDE

[Stefan Podkowinski (JIRA)]

Stefan Podkowinski created CASSANDRA-14107:
--

 Summary: Introduce simple key alias versioning scheme for TDE
 Key: CASSANDRA-14107
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14107
 Project: Cassandra
  Issue Type: New Feature
Reporter: Stefan Podkowinski
Priority: Minor


Handling of encryption keys as introduced in CASSANDRA-9945 takes place by 
referencing a key alias in either cassandra.yaml, or the header of the 
(commitlog/hints) file that has been encrypted. Using the alias as literal 
value will work, but requires some attention when rotating keys.

Currently each time a key is rotated (i.e. adding a new key to the keystore 
while preserving the previous version), the alias in cassandra.yaml has to be 
update as well and the node needs to be restarted. It would be more convenient 
to use a symbolic reference instead. My suggestion here would be to use 
":last" for referring to the latest version. Omitting the ":" 
part altogether would have the same effect. In this case Cassandra always picks 
the latest key when the key cache has been expired.

The non-trivial part of this suggestion is how the "latest" key is referenced 
in the file header. If we use "latest", e.g. for the commit log header, and the 
key gets rotated, we'd now try do decrypt the file with the new key, instead of 
the key it has been created with. Therefor we'd have to introduce an extra step 
that will resolve the canonical version for "latest" and refer to that one 
during any encrypt operation. 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14107) Introduce simple key alias versioning scheme for TDE

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-14107:
---
Fix Version/s: 4.x

> Introduce simple key alias versioning scheme for TDE
> 
>
> Key: CASSANDRA-14107
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14107
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Stefan Podkowinski
>Priority: Minor
>  Labels: encryption
> Fix For: 4.x
>
>
> Handling of encryption keys as introduced in CASSANDRA-9945 takes place by 
> referencing a key alias in either cassandra.yaml, or the header of the 
> (commitlog/hints) file that has been encrypted. Using the alias as literal 
> value will work, but requires some attention when rotating keys.
> Currently each time a key is rotated (i.e. adding a new key to the keystore 
> while preserving the previous version), the alias in cassandra.yaml has to be 
> update as well and the node needs to be restarted. It would be more 
> convenient to use a symbolic reference instead. My suggestion here would be 
> to use ":last" for referring to the latest version. Omitting the 
> ":" part altogether would have the same effect. In this case 
> Cassandra always picks the latest key when the key cache has been expired.
> The non-trivial part of this suggestion is how the "latest" key is referenced 
> in the file header. If we use "latest", e.g. for the commit log header, and 
> the key gets rotated, we'd now try do decrypt the file with the new key, 
> instead of the key it has been created with. Therefor we'd have to introduce 
> an extra step that will resolve the canonical version for "latest" and refer 
> to that one during any encrypt operation. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13426) Make all DDL statements idempotent and not dependent on global state

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287636#comment-16287636
 ] 

Sam Tunnicliffe commented on CASSANDRA-13426:
-

I've been looking at the WIP branch for a little while now. Looks good, lots of 
great improvements. I have a few, mostly minor comments:

* AlterSchemaStatement
** In grantPermissionsOnResource can use AuthenticatedUser::getPrimaryRole
* QueryProcessor
** #402 assert is redundant
** #422 typo "MDnt5"
** Should Prepared belong to QueryHandler rather than QP?
* ClientState
** ensureIsSuperUser is the only place we camel case SuperUser
* TableMetadata/TableParams/Views
** Maybe a bit subjective, but I find the naming of "unbuild" unintuitive. How 
about something like builderFrom/makeBuilder/asBuilder/toBuilder ?
* Views
** metadatas() method should be renamed - data is already plural, but also 
could it be named to reflect the fact that it returns an 
Iterable, whilst Views itself implements Iterable. 
It's a bit ambiguous as it is.
* TableId
** Typo in javadoc: s/nicely name class make/nicely named class makes
* ViewMetadata
** equals uses WhereClause::equals, but this isn't overridden
** the comment in withRenamedPrimaryKeyColumns belongs in 
WhereClause::renameIdentifier, if it's actually necessary
* TableMetadata
** toDebugString is unused
* Tables
** getNullable is missing annotation (all other similar methods have it).
* Keyspaces
** get(String name) is unused
* KeyspaceDiff
** A comment explaining why function diffs need to be handled differently could 
be useful (I'm assuming it's because the filtering to distinguish UDFs/UDAs 
makes it slightly more expensive than the other types of diff).
* Functions
** can Filter.test use isAggregate rather than instanceof ?
** aggregatesUsingFunction, comment should reads s/collection/stream
* CompressionParams
** outstanding TODO on setCrcCheckChance
* SetType/ListType/MapType/DynamicCompositeType
** getInstance can be simplified to just return the result of computeIfAbsent. 
There's an unchecked warning, but I'm not sure that's any different from the 
existing one. ReversedType and CompositeType impls already do this.
* AbstractType
** I think the comment on withUpdatedUserType could be a little clearer. Maybe 
something like "Returns an instance with all references to the supplied 
UserTypei recursively replaced with its new definition".
* SelectStatement
** The order of constructor args was changed to swap TableMetadata & the bind 
variables. NBD, but having the TM first seems more logical to me, was there a 
reason behind the switch?
* QualifiedName
** Typo in class level javadoc - s/CLass/Class
* StatementRestrictions
** The comment on the new ctor is slightly incorrect. We want to override the 
allow 2i flag from the StatementType for MV statements to avoid initing the the 
KS and SecondaryIndexManager. We don't open the KS to determine the value of 
the allow flag in either case.
* UDAggregate
** When reconstructing from schema tables and the function can't be 
reconstructed for whatever reason - we preserve the old behaviour with a dummy, 
broken function for UDFs but not for UDAs. These now trigger an assert. Is this 
an issue?
* DynamicCompositeType
** needs an expandUserTypes impl?
** withUpdatedUserType won't work correctly now that referencesUserType is 
inherited from AbstractType
* UserType
** expandUserType override isn't necessary, though I'm OK with leaving it in 
for clarity
* MigrationManager
** Comment on forceAnnounceNewTable - 
s/announceUpdateColumnFamily/announceTableUpdate
* CreateViewStatement
** validation is now stricter in that it doesn't silently accept SELECT that 
doesn't contain all primary key elements, or PK columns without IS NOT NULL. 
It's good, just noting that this should be documented somewhere.
* UFTest
** line #114 seems to have a typo - s/string1/string
* VariableSpecifications/CQLStatement
** there's a minor inconsistency in method naming here: 
getPartitionKeyBindIndexes vs getPartitionKeyBindVariableIndexes.


> Make all DDL statements idempotent and not dependent on global state
> 
>
> Key: CASSANDRA-13426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13426
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 4.0
>
>
> A follow-up to CASSANDRA-9425 and a pre-requisite for CASSANDRA-10699.
> It's necessary for the latter to be able to apply any DDL statement several 
> times without side-effects. As part of the ticket I think we should also 
> clean up validation logic and our error texts. One example is varying 
> treatment of missing keyspace for DROP TABLE/INDEX/etc. statements with IF 
> 

[jira] [Commented] (CASSANDRA-13426) Make all DDL statements idempotent and not dependent on global state

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287526#comment-16287526
 ] 

Aleksey Yeschenko commented on CASSANDRA-13426:
---

Force-pushed a rebased branch with some updates. At this point it's largely 
done, missing some optimisations on shallow vs. deep diffs for delta 
serialisation, and good test coverage.

> Make all DDL statements idempotent and not dependent on global state
> 
>
> Key: CASSANDRA-13426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13426
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 4.0
>
>
> A follow-up to CASSANDRA-9425 and a pre-requisite for CASSANDRA-10699.
> It's necessary for the latter to be able to apply any DDL statement several 
> times without side-effects. As part of the ticket I think we should also 
> clean up validation logic and our error texts. One example is varying 
> treatment of missing keyspace for DROP TABLE/INDEX/etc. statements with IF 
> EXISTS.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14008) RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287517#comment-16287517
 ] 

Aleksey Yeschenko commented on CASSANDRA-14008:
---

Validated the test, both logically, and by running {{LegacyLayoutTest}} in both 
branches, with and without the fix reverted, ensuring that it doesn't fail, 
then fails, respectively.

+1

> RTs at index boundaries in 2.x sstables can create unexpected CQL row in 3.x
> 
>
> Key: CASSANDRA-14008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14008
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>  Labels: correctness
> Fix For: 3.0.16, 3.11.2
>
>
> In 2.1/2.2, it is possible for a range tombstone that isn't a row deletion 
> and isn't a complex deletion to appear between two cells with the same 
> clustering. The 8099 legacy code incorrectly treats the two (non-RT) cells as 
> two distinct CQL rows, despite having the same clustering prefix.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13006) Disable automatic heap dumps on OOM error

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13006:
---
   Resolution: Fixed
Fix Version/s: 4.0
   3.11.2
   3.0.16
   2.2.12
   Status: Resolved  (was: Ready to Commit)

Committed into cassandra-2.2 at 02aba7343ce300397ab672bbb1788aa8182d8a48 and 
merged into 3.0, 3.11 and trunk

> Disable automatic heap dumps on OOM error
> -
>
> Key: CASSANDRA-13006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13006
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: anmols
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.12, 3.0.16, 3.11.2, 4.0
>
> Attachments: 13006-3.0.9.txt
>
>
> With CASSANDRA-9861, a change was added to enable collecting heap dumps by 
> default if the process encountered an OOM error. These heap dumps are stored 
> in the Apache Cassandra home directory unless configured otherwise (see 
> [Cassandra Support 
> Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps]
>  for this feature).
>  
> The creation and storage of heap dumps aides debugging and investigative 
> workflows, but is not be desirable for a production environment where these 
> heap dumps may occupy a large amount of disk space and require manual 
> intervention for cleanups. 
>  
> Managing heap dumps on out of memory errors and configuring the paths for 
> these heap dumps are available as JVM options in JVM. The current behavior 
> conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. 
>  
> A patch can be proposed here that would make the heap dump on OOM error honor 
> the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate 
> heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM 
> option.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[07/10] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd187d10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd187d10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd187d10

Branch: refs/heads/trunk
Commit: dd187d105bce300395f52eb8ca4d8ccaf94a06ea
Parents: d7329a6 02aba73
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:30:29 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:32:36 2017 +0100

--
 CHANGES.txt |   1 +
 NEWS.txt|   9 +-
 bin/cassandra   |  19 +-
 conf/cassandra-env.ps1  | 950 ++-
 conf/cassandra-env.sh   |  12 +
 .../apache/cassandra/service/StartupChecks.java |  74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   |  93 +-
 .../cassandra/utils/JVMStabilityInspector.java  |  24 +-
 .../utils/JVMStabilityInspectorTest.java|  28 +-
 9 files changed, 650 insertions(+), 560 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd187d10/CHANGES.txt
--
diff --cc CHANGES.txt
index 8cff716,5200eb1..20ccc4b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,83 -1,8 +1,84 @@@
 -2.2.12
 +3.0.16
 + * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
 + * Accept role names containing forward-slash (CASSANDRA-14088)
 + * Optimize CRC check chance probability calculations (CASSANDRA-14094)
 + * Fix cleanup on keyspace with no replicas (CASSANDRA-13526)
 + * Fix updating base table rows with TTL not removing materialized view 
entries (CASSANDRA-14071)
 + * Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
 + * More frequent commitlog chained markers (CASSANDRA-13987)
 + * Fix serialized size of DataLimits (CASSANDRA-14057)
 + * Add flag to allow dropping oversized read repair mutations 
(CASSANDRA-13975)
 + * Fix SSTableLoader logger message (CASSANDRA-14003)
 + * Fix repair race that caused gossip to block (CASSANDRA-13849)
 + * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
 + * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
 + * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)
 + * Provide a JMX call to sync schema with local storage (CASSANDRA-13954)
 + * Mishandling of cells for removed/dropped columns when reading legacy files 
(CASSANDRA-13939)
 + * Deserialise sstable metadata in nodetool verify (CASSANDRA-13922)
 +Merged from 2.2:
+  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
   * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
  
 -2.2.11
 +
 +3.0.15
 + * Improve TRUNCATE performance (CASSANDRA-13909)
 + * Implement short read protection on partition boundaries (CASSANDRA-13595)
 + * Fix ISE thrown by UPI.Serializer.hasNext() for some SELECT queries 
(CASSANDRA-13911)
 + * Filter header only commit logs before recovery (CASSANDRA-13918)
 + * AssertionError prepending to a list (CASSANDRA-13149)
 + * Fix support for SuperColumn tables (CASSANDRA-12373)
 + * Handle limit correctly on tables with strict liveness (CASSANDRA-13883)
 + * Fix missing original update in TriggerExecutor (CASSANDRA-13894)
 + * Remove non-rpc-ready nodes from counter leader candidates (CASSANDRA-13043)
 + * Improve short read protection performance (CASSANDRA-13794)
 + * Fix sstable reader to support range-tombstone-marker for multi-slices 
(CASSANDRA-13787)
 + * Fix short read protection for tables with no clustering columns 
(CASSANDRA-13880)
 + * Make isBuilt volatile in PartitionUpdate (CASSANDRA-13619)
 + * Prevent integer overflow of timestamps in CellTest and RowsTest 
(CASSANDRA-13866)
 + * Fix counter application order in short read protection (CASSANDRA-12872)
 + * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
 + * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
 + * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
 + * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
 + * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)
 + * Range deletes in a CAS batch are ignored (CASSANDRA-13655)
 + * Avoid assertion error when IndexSummary > 2G (CASSANDRA-12014)
 + * Change repair midpoint logging for tiny ranges (CASSANDRA-13603)
 + * Better handle corrupt final commitlog segment (CASSANDRA-11995)
 + * 

[03/10] cassandra git commit: Rely on the JVM to handle OutOfMemoryErrors

[blerer]

Rely on the JVM to handle OutOfMemoryErrors

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-13006


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/02aba734
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/02aba734
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/02aba734

Branch: refs/heads/cassandra-3.11
Commit: 02aba7343ce300397ab672bbb1788aa8182d8a48
Parents: 3cd2c3c
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:21:05 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:21:05 2017 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  9 ++-
 bin/cassandra   | 19 -
 conf/cassandra-env.ps1  | 10 +++
 conf/cassandra-env.sh   | 12 +++
 .../apache/cassandra/service/StartupChecks.java | 74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   | 82 
 .../cassandra/utils/JVMStabilityInspector.java  | 24 +-
 .../utils/JVMStabilityInspectorTest.java| 31 +---
 9 files changed, 178 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c1e81fd..5200eb1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 
 2.2.11

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 3bff458..5747941 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,8 +18,13 @@ using the provided 'sstableupgrade' tool.
 
 Upgrading
 -
-- Nothing specific to this release, but please see 2.2 if you are upgrading
-  from a previous version.
+- Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
+  rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
+  are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
+  options are used. See CASSANDRA-13006 for more details.
+- Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
+  set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property 
to 'true'. See CASSANDRA-13006
+  for more details.
 
 2.2.11
 ==

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index 2dd0fe1..0e337e8 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -28,6 +28,7 @@
 #
 #   CLASSPATH -- A Java classpath containing everything necessary to run.
 #   JVM_OPTS -- Additional arguments to the JVM for heap size, etc
+#   JVM_ON_OUT_OF_MEMORY_ERROR_OPT -- The OnOutOfMemoryError JVM option if 
specified
 #   CASSANDRA_CONF -- Directory containing Cassandra configuration files.
 #
 # As a convenience, a fragment of shell is sourced in order to set one or
@@ -199,12 +200,22 @@ launch_service()
 # to close stdout/stderr, but it's up to us not to background.
 if [ "x$foreground" != "x" ]; then
 cassandra_parms="$cassandra_parms -Dcassandra-foreground=yes"
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class"
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+fi
 # Startup CassandraDaemon, background it, and write the pid.
 else
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
-[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
-true
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+true
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+true
+fi
 fi

[06/10] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd187d10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd187d10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd187d10

Branch: refs/heads/cassandra-3.0
Commit: dd187d105bce300395f52eb8ca4d8ccaf94a06ea
Parents: d7329a6 02aba73
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:30:29 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:32:36 2017 +0100

--
 CHANGES.txt |   1 +
 NEWS.txt|   9 +-
 bin/cassandra   |  19 +-
 conf/cassandra-env.ps1  | 950 ++-
 conf/cassandra-env.sh   |  12 +
 .../apache/cassandra/service/StartupChecks.java |  74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   |  93 +-
 .../cassandra/utils/JVMStabilityInspector.java  |  24 +-
 .../utils/JVMStabilityInspectorTest.java|  28 +-
 9 files changed, 650 insertions(+), 560 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd187d10/CHANGES.txt
--
diff --cc CHANGES.txt
index 8cff716,5200eb1..20ccc4b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,83 -1,8 +1,84 @@@
 -2.2.12
 +3.0.16
 + * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
 + * Accept role names containing forward-slash (CASSANDRA-14088)
 + * Optimize CRC check chance probability calculations (CASSANDRA-14094)
 + * Fix cleanup on keyspace with no replicas (CASSANDRA-13526)
 + * Fix updating base table rows with TTL not removing materialized view 
entries (CASSANDRA-14071)
 + * Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
 + * More frequent commitlog chained markers (CASSANDRA-13987)
 + * Fix serialized size of DataLimits (CASSANDRA-14057)
 + * Add flag to allow dropping oversized read repair mutations 
(CASSANDRA-13975)
 + * Fix SSTableLoader logger message (CASSANDRA-14003)
 + * Fix repair race that caused gossip to block (CASSANDRA-13849)
 + * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
 + * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
 + * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)
 + * Provide a JMX call to sync schema with local storage (CASSANDRA-13954)
 + * Mishandling of cells for removed/dropped columns when reading legacy files 
(CASSANDRA-13939)
 + * Deserialise sstable metadata in nodetool verify (CASSANDRA-13922)
 +Merged from 2.2:
+  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
   * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
  
 -2.2.11
 +
 +3.0.15
 + * Improve TRUNCATE performance (CASSANDRA-13909)
 + * Implement short read protection on partition boundaries (CASSANDRA-13595)
 + * Fix ISE thrown by UPI.Serializer.hasNext() for some SELECT queries 
(CASSANDRA-13911)
 + * Filter header only commit logs before recovery (CASSANDRA-13918)
 + * AssertionError prepending to a list (CASSANDRA-13149)
 + * Fix support for SuperColumn tables (CASSANDRA-12373)
 + * Handle limit correctly on tables with strict liveness (CASSANDRA-13883)
 + * Fix missing original update in TriggerExecutor (CASSANDRA-13894)
 + * Remove non-rpc-ready nodes from counter leader candidates (CASSANDRA-13043)
 + * Improve short read protection performance (CASSANDRA-13794)
 + * Fix sstable reader to support range-tombstone-marker for multi-slices 
(CASSANDRA-13787)
 + * Fix short read protection for tables with no clustering columns 
(CASSANDRA-13880)
 + * Make isBuilt volatile in PartitionUpdate (CASSANDRA-13619)
 + * Prevent integer overflow of timestamps in CellTest and RowsTest 
(CASSANDRA-13866)
 + * Fix counter application order in short read protection (CASSANDRA-12872)
 + * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
 + * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
 + * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
 + * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
 + * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)
 + * Range deletes in a CAS batch are ignored (CASSANDRA-13655)
 + * Avoid assertion error when IndexSummary > 2G (CASSANDRA-12014)
 + * Change repair midpoint logging for tiny ranges (CASSANDRA-13603)
 + * Better handle corrupt final commitlog segment (CASSANDRA-11995)
 + 

[09/10] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7c45be9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7c45be9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7c45be9

Branch: refs/heads/trunk
Commit: a7c45be936bf8aa04b657529964c41b0d82c50e2
Parents: 88a41fb dd187d1
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:38:33 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:42:50 2017 +0100

--
 CHANGES.txt |   2 +-
 NEWS.txt|   8 +-
 bin/cassandra   |  19 +-
 conf/cassandra-env.ps1  | 914 ++-
 conf/cassandra-env.sh   |  12 +
 .../apache/cassandra/service/StartupChecks.java |  74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   |  93 +-
 .../cassandra/utils/JVMStabilityInspector.java  |  24 +-
 .../utils/JVMStabilityInspectorTest.java|  28 +-
 9 files changed, 632 insertions(+), 542 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/CHANGES.txt
--
diff --cc CHANGES.txt
index 6e9a0bd,20ccc4b..116c7e9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,4 +1,14 @@@
 -3.0.16
 +3.11.2
 + * Fix imbalanced disks when replacing node with same address with JBOD 
(CASSANDRA-14084)
 + * Reload compaction strategies when disk boundaries are invalidated 
(CASSANDRA-13948)
 + * Remove OpenJDK log warning (CASSANDRA-13916)
 + * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079)
 + * Cache disk boundaries (CASSANDRA-13215)
 + * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
 + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
 + * Update jackson JSON jars (CASSANDRA-13949)
 + * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)
 +Merged from 3.0:
- 3.0.16
   * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
   * Accept role names containing forward-slash (CASSANDRA-14088)
   * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/NEWS.txt
--
diff --cc NEWS.txt
index 9812b22,1bbe1aa..0c32278
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -18,7 -18,13 +18,13 @@@ using the provided 'sstableupgrade' too
  
  Upgrading
  -
- - Nothing specific to this release, but please see previous upgrading 
sections.
 -   - Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
 - rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
 - are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
 - options are used. See CASSANDRA-13006 for more details.
 -   - Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
 - set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property 
to 'true'. See CASSANDRA-13006
 - for more details.
++- Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
++  rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
++  are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
++  options are used. See CASSANDRA-13006 for more details
++- Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
++  set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System 
property to 'true'. See CASSANDRA-13006
++  for more details.
  
  Materialized Views
  ---

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/bin/cassandra
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/conf/cassandra-env.ps1
--
diff --cc conf/cassandra-env.ps1
index 2df7054,405ed92..49d03ce
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@@ -393,70 -393,98 +393,80 @@@ Function SetCassandraEnvironmen
  $env:JVM_OPTS="$env:JVM_OPTS 
-XX:HeapDumpPath=$env:CASSANDRA_HEAPDUMP_DIR\cassandra-$unixTimestamp-pid$pid.hprof"
  }
  
- if ($env:JVM_VERSION.CompareTo("1.8.0") -eq -1 -or 
[convert]::ToInt32($env:JVM_PATCH_VERSION) 

[05/10] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd187d10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd187d10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd187d10

Branch: refs/heads/cassandra-3.11
Commit: dd187d105bce300395f52eb8ca4d8ccaf94a06ea
Parents: d7329a6 02aba73
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:30:29 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:32:36 2017 +0100

--
 CHANGES.txt |   1 +
 NEWS.txt|   9 +-
 bin/cassandra   |  19 +-
 conf/cassandra-env.ps1  | 950 ++-
 conf/cassandra-env.sh   |  12 +
 .../apache/cassandra/service/StartupChecks.java |  74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   |  93 +-
 .../cassandra/utils/JVMStabilityInspector.java  |  24 +-
 .../utils/JVMStabilityInspectorTest.java|  28 +-
 9 files changed, 650 insertions(+), 560 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd187d10/CHANGES.txt
--
diff --cc CHANGES.txt
index 8cff716,5200eb1..20ccc4b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,83 -1,8 +1,84 @@@
 -2.2.12
 +3.0.16
 + * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
 + * Accept role names containing forward-slash (CASSANDRA-14088)
 + * Optimize CRC check chance probability calculations (CASSANDRA-14094)
 + * Fix cleanup on keyspace with no replicas (CASSANDRA-13526)
 + * Fix updating base table rows with TTL not removing materialized view 
entries (CASSANDRA-14071)
 + * Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
 + * More frequent commitlog chained markers (CASSANDRA-13987)
 + * Fix serialized size of DataLimits (CASSANDRA-14057)
 + * Add flag to allow dropping oversized read repair mutations 
(CASSANDRA-13975)
 + * Fix SSTableLoader logger message (CASSANDRA-14003)
 + * Fix repair race that caused gossip to block (CASSANDRA-13849)
 + * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
 + * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
 + * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)
 + * Provide a JMX call to sync schema with local storage (CASSANDRA-13954)
 + * Mishandling of cells for removed/dropped columns when reading legacy files 
(CASSANDRA-13939)
 + * Deserialise sstable metadata in nodetool verify (CASSANDRA-13922)
 +Merged from 2.2:
+  * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
   * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
  
 -2.2.11
 +
 +3.0.15
 + * Improve TRUNCATE performance (CASSANDRA-13909)
 + * Implement short read protection on partition boundaries (CASSANDRA-13595)
 + * Fix ISE thrown by UPI.Serializer.hasNext() for some SELECT queries 
(CASSANDRA-13911)
 + * Filter header only commit logs before recovery (CASSANDRA-13918)
 + * AssertionError prepending to a list (CASSANDRA-13149)
 + * Fix support for SuperColumn tables (CASSANDRA-12373)
 + * Handle limit correctly on tables with strict liveness (CASSANDRA-13883)
 + * Fix missing original update in TriggerExecutor (CASSANDRA-13894)
 + * Remove non-rpc-ready nodes from counter leader candidates (CASSANDRA-13043)
 + * Improve short read protection performance (CASSANDRA-13794)
 + * Fix sstable reader to support range-tombstone-marker for multi-slices 
(CASSANDRA-13787)
 + * Fix short read protection for tables with no clustering columns 
(CASSANDRA-13880)
 + * Make isBuilt volatile in PartitionUpdate (CASSANDRA-13619)
 + * Prevent integer overflow of timestamps in CellTest and RowsTest 
(CASSANDRA-13866)
 + * Fix counter application order in short read protection (CASSANDRA-12872)
 + * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
 + * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
 + * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
 + * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
 + * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)
 + * Range deletes in a CAS batch are ignored (CASSANDRA-13655)
 + * Avoid assertion error when IndexSummary > 2G (CASSANDRA-12014)
 + * Change repair midpoint logging for tiny ranges (CASSANDRA-13603)
 + * Better handle corrupt final commitlog segment (CASSANDRA-11995)
 + 

[08/10] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7c45be9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7c45be9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7c45be9

Branch: refs/heads/cassandra-3.11
Commit: a7c45be936bf8aa04b657529964c41b0d82c50e2
Parents: 88a41fb dd187d1
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:38:33 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:42:50 2017 +0100

--
 CHANGES.txt |   2 +-
 NEWS.txt|   8 +-
 bin/cassandra   |  19 +-
 conf/cassandra-env.ps1  | 914 ++-
 conf/cassandra-env.sh   |  12 +
 .../apache/cassandra/service/StartupChecks.java |  74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   |  93 +-
 .../cassandra/utils/JVMStabilityInspector.java  |  24 +-
 .../utils/JVMStabilityInspectorTest.java|  28 +-
 9 files changed, 632 insertions(+), 542 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/CHANGES.txt
--
diff --cc CHANGES.txt
index 6e9a0bd,20ccc4b..116c7e9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,4 +1,14 @@@
 -3.0.16
 +3.11.2
 + * Fix imbalanced disks when replacing node with same address with JBOD 
(CASSANDRA-14084)
 + * Reload compaction strategies when disk boundaries are invalidated 
(CASSANDRA-13948)
 + * Remove OpenJDK log warning (CASSANDRA-13916)
 + * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079)
 + * Cache disk boundaries (CASSANDRA-13215)
 + * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
 + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
 + * Update jackson JSON jars (CASSANDRA-13949)
 + * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)
 +Merged from 3.0:
- 3.0.16
   * Fix SStable ordering by max timestamp in SinglePartitionReadCommand 
(CASSANDRA-14010)
   * Accept role names containing forward-slash (CASSANDRA-14088)
   * Optimize CRC check chance probability calculations (CASSANDRA-14094)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/NEWS.txt
--
diff --cc NEWS.txt
index 9812b22,1bbe1aa..0c32278
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -18,7 -18,13 +18,13 @@@ using the provided 'sstableupgrade' too
  
  Upgrading
  -
- - Nothing specific to this release, but please see previous upgrading 
sections.
 -   - Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
 - rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
 - are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
 - options are used. See CASSANDRA-13006 for more details.
 -   - Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
 - set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property 
to 'true'. See CASSANDRA-13006
 - for more details.
++- Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
++  rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
++  are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
++  options are used. See CASSANDRA-13006 for more details
++- Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
++  set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System 
property to 'true'. See CASSANDRA-13006
++  for more details.
  
  Materialized Views
  ---

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/bin/cassandra
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/conf/cassandra-env.ps1
--
diff --cc conf/cassandra-env.ps1
index 2df7054,405ed92..49d03ce
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@@ -393,70 -393,98 +393,80 @@@ Function SetCassandraEnvironmen
  $env:JVM_OPTS="$env:JVM_OPTS 
-XX:HeapDumpPath=$env:CASSANDRA_HEAPDUMP_DIR\cassandra-$unixTimestamp-pid$pid.hprof"
  }
  
- if ($env:JVM_VERSION.CompareTo("1.8.0") -eq -1 -or 

[02/10] cassandra git commit: Rely on the JVM to handle OutOfMemoryErrors

[blerer]

Rely on the JVM to handle OutOfMemoryErrors

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-13006


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/02aba734
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/02aba734
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/02aba734

Branch: refs/heads/cassandra-3.0
Commit: 02aba7343ce300397ab672bbb1788aa8182d8a48
Parents: 3cd2c3c
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:21:05 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:21:05 2017 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  9 ++-
 bin/cassandra   | 19 -
 conf/cassandra-env.ps1  | 10 +++
 conf/cassandra-env.sh   | 12 +++
 .../apache/cassandra/service/StartupChecks.java | 74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   | 82 
 .../cassandra/utils/JVMStabilityInspector.java  | 24 +-
 .../utils/JVMStabilityInspectorTest.java| 31 +---
 9 files changed, 178 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c1e81fd..5200eb1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 
 2.2.11

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 3bff458..5747941 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,8 +18,13 @@ using the provided 'sstableupgrade' tool.
 
 Upgrading
 -
-- Nothing specific to this release, but please see 2.2 if you are upgrading
-  from a previous version.
+- Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
+  rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
+  are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
+  options are used. See CASSANDRA-13006 for more details.
+- Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
+  set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property 
to 'true'. See CASSANDRA-13006
+  for more details.
 
 2.2.11
 ==

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index 2dd0fe1..0e337e8 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -28,6 +28,7 @@
 #
 #   CLASSPATH -- A Java classpath containing everything necessary to run.
 #   JVM_OPTS -- Additional arguments to the JVM for heap size, etc
+#   JVM_ON_OUT_OF_MEMORY_ERROR_OPT -- The OnOutOfMemoryError JVM option if 
specified
 #   CASSANDRA_CONF -- Directory containing Cassandra configuration files.
 #
 # As a convenience, a fragment of shell is sourced in order to set one or
@@ -199,12 +200,22 @@ launch_service()
 # to close stdout/stderr, but it's up to us not to background.
 if [ "x$foreground" != "x" ]; then
 cassandra_parms="$cassandra_parms -Dcassandra-foreground=yes"
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class"
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+fi
 # Startup CassandraDaemon, background it, and write the pid.
 else
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
-[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
-true
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+true
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+true
+fi
 fi
 

[01/10] cassandra git commit: Rely on the JVM to handle OutOfMemoryErrors

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 3cd2c3c4e -> 02aba7343
  refs/heads/cassandra-3.0 d7329a639 -> dd187d105
  refs/heads/cassandra-3.11 88a41fb82 -> a7c45be93
  refs/heads/trunk 8e95534fb -> 17e602b7d


Rely on the JVM to handle OutOfMemoryErrors

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-13006


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/02aba734
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/02aba734
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/02aba734

Branch: refs/heads/cassandra-2.2
Commit: 02aba7343ce300397ab672bbb1788aa8182d8a48
Parents: 3cd2c3c
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:21:05 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:21:05 2017 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  9 ++-
 bin/cassandra   | 19 -
 conf/cassandra-env.ps1  | 10 +++
 conf/cassandra-env.sh   | 12 +++
 .../apache/cassandra/service/StartupChecks.java | 74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   | 82 
 .../cassandra/utils/JVMStabilityInspector.java  | 24 +-
 .../utils/JVMStabilityInspectorTest.java| 31 +---
 9 files changed, 178 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c1e81fd..5200eb1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 
 2.2.11

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 3bff458..5747941 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,8 +18,13 @@ using the provided 'sstableupgrade' tool.
 
 Upgrading
 -
-- Nothing specific to this release, but please see 2.2 if you are upgrading
-  from a previous version.
+- Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
+  rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
+  are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
+  options are used. See CASSANDRA-13006 for more details.
+- Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
+  set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property 
to 'true'. See CASSANDRA-13006
+  for more details.
 
 2.2.11
 ==

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index 2dd0fe1..0e337e8 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -28,6 +28,7 @@
 #
 #   CLASSPATH -- A Java classpath containing everything necessary to run.
 #   JVM_OPTS -- Additional arguments to the JVM for heap size, etc
+#   JVM_ON_OUT_OF_MEMORY_ERROR_OPT -- The OnOutOfMemoryError JVM option if 
specified
 #   CASSANDRA_CONF -- Directory containing Cassandra configuration files.
 #
 # As a convenience, a fragment of shell is sourced in order to set one or
@@ -199,12 +200,22 @@ launch_service()
 # to close stdout/stderr, but it's up to us not to background.
 if [ "x$foreground" != "x" ]; then
 cassandra_parms="$cassandra_parms -Dcassandra-foreground=yes"
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class"
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+fi
 # Startup CassandraDaemon, background it, and write the pid.
 else
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
-[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
-true
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+ 

[04/10] cassandra git commit: Rely on the JVM to handle OutOfMemoryErrors

[blerer]

Rely on the JVM to handle OutOfMemoryErrors

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-13006


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/02aba734
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/02aba734
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/02aba734

Branch: refs/heads/trunk
Commit: 02aba7343ce300397ab672bbb1788aa8182d8a48
Parents: 3cd2c3c
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:21:05 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:21:05 2017 +0100

--
 CHANGES.txt |  1 +
 NEWS.txt|  9 ++-
 bin/cassandra   | 19 -
 conf/cassandra-env.ps1  | 10 +++
 conf/cassandra-env.sh   | 12 +++
 .../apache/cassandra/service/StartupChecks.java | 74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   | 82 
 .../cassandra/utils/JVMStabilityInspector.java  | 24 +-
 .../utils/JVMStabilityInspectorTest.java| 31 +---
 9 files changed, 178 insertions(+), 84 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c1e81fd..5200eb1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.12
+ * Rely on the JVM to handle OutOfMemoryErrors (CASSANDRA-13006)
  * Grab refs during scrub/index redistribution/cleanup (CASSANDRA-13873)
 
 2.2.11

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 3bff458..5747941 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,8 +18,13 @@ using the provided 'sstableupgrade' tool.
 
 Upgrading
 -
-- Nothing specific to this release, but please see 2.2 if you are upgrading
-  from a previous version.
+- Cassandra is now relying on the JVM options to properly shutdown on 
OutOfMemoryError. By default it will
+  rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and 
CrashOnOutOfMemoryError options
+  are not supported by the older 1.7 and 1.8 JVMs. A warning will be 
logged at startup if none of those JVM
+  options are used. See CASSANDRA-13006 for more details.
+- Cassandra is not logging anymore by default an Heap histogram on 
OutOfMemoryError. To enable that behavior
+  set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property 
to 'true'. See CASSANDRA-13006
+  for more details.
 
 2.2.11
 ==

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02aba734/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index 2dd0fe1..0e337e8 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -28,6 +28,7 @@
 #
 #   CLASSPATH -- A Java classpath containing everything necessary to run.
 #   JVM_OPTS -- Additional arguments to the JVM for heap size, etc
+#   JVM_ON_OUT_OF_MEMORY_ERROR_OPT -- The OnOutOfMemoryError JVM option if 
specified
 #   CASSANDRA_CONF -- Directory containing Cassandra configuration files.
 #
 # As a convenience, a fragment of shell is sourced in order to set one or
@@ -199,12 +200,22 @@ launch_service()
 # to close stdout/stderr, but it's up to us not to background.
 if [ "x$foreground" != "x" ]; then
 cassandra_parms="$cassandra_parms -Dcassandra-foreground=yes"
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class"
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class"
+fi
 # Startup CassandraDaemon, background it, and write the pid.
 else
-exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
-[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
-true
+if [ "x$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" != "x" ]; then
+exec $NUMACTL "$JAVA" $JVM_OPTS "$JVM_ON_OUT_OF_MEMORY_ERROR_OPT" 
$cassandra_parms -cp "$CLASSPATH" $props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+true
+else
+exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" 
$props "$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
+true
+fi
 fi
 
 

[10/10] cassandra git commit: Merge branch cassandra-3.11 into trunk

[blerer]

Merge branch cassandra-3.11 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17e602b7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17e602b7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17e602b7

Branch: refs/heads/trunk
Commit: 17e602b7d248c61634c31c93b7d244f35cfcefd3
Parents: 8e95534 a7c45be
Author: Benjamin Lerer 
Authored: Tue Dec 12 10:52:23 2017 +0100
Committer: Benjamin Lerer 
Committed: Tue Dec 12 10:52:23 2017 +0100

--
 CHANGES.txt |   2 +-
 NEWS.txt|  13 +
 bin/cassandra   |  19 +-
 conf/cassandra-env.ps1  | 914 ++-
 conf/cassandra-env.sh   |  12 +
 .../apache/cassandra/service/StartupChecks.java |  74 ++
 .../org/apache/cassandra/utils/HeapUtils.java   |  93 +-
 .../cassandra/utils/JVMStabilityInspector.java  |  24 +-
 .../utils/JVMStabilityInspectorTest.java|  28 +-
 9 files changed, 638 insertions(+), 541 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/NEWS.txt
--
diff --cc NEWS.txt
index 7bb9152,0c32278..259507e
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,103 -13,27 +13,116 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
  
 +4.0
 +===
 +
 +New features
 +
 +   - An experimental option to compare all merkle trees together has been 
added - for example, in
 + a 3 node cluster with 2 replicas identical and 1 out-of-date, with this 
option enabled, the
 + out-of-date replica will only stream a single copy from up-to-date 
replica. Enable it by adding
 + "-os" to nodetool repair. See CASSANDRA-3200.
 +   - The currentTimestamp, currentDate, currentTime and currentTimeUUID 
functions have been added.
 + See CASSANDRA-13132
 +   - Support for arithmetic operations between `timestamp`/`date` and 
`duration` has been added.
 + See CASSANDRA-11936
 +   - Support for arithmetic operations on number has been added. See 
CASSANDRA-11935
 +   - Preview expected streaming required for a repair (nodetool repair 
--preview), and validate the
 + consistency of repaired data between nodes (nodetool repair --validate). 
See CASSANDRA-13257
 +   - Support for selecting Map values and Set elements has been added for 
SELECT queries. See CASSANDRA-7396
 +   - Change-Data-Capture has been modified to make CommitLogSegments available
 + immediately upon creation via hard-linking the files. This means that 
incomplete
 + segments will be available in cdc_raw rather than fully flushed. See 
documentation
 + and CASSANDRA-12148 for more detail.
 +   - The initial build of materialized views can be parallelized. The number 
of concurrent builder
 + threads is specified by the property 
`cassandra.yaml:concurrent_materialized_view_builders`.
 + This property can be modified at runtime through both JMX and the new 
`setconcurrentviewbuilders`
 + and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 
for more details.
 +   - There is now a binary full query log based on Chronicle Queue that can 
be controlled using
 + nodetool enablefullquerylog, disablefullquerylog, and resetfullquerylog. 
The log 
 + contains all queries invoked, approximate time they were invoked, any 
parameters necessary
 + to bind wildcard values, and all query options. A human readable version 
of the log can be
 + dumped or tailed using the new bin/fqltool utility. The full query log 
is designed to be safe
 + to use in production and limits utilization of heap memory and disk 
space with limits
 + you can specify when enabling the log.
 + See nodetool and fqltool help text for more information.
 +   - SSTableDump now supports the -l option to output each partition as it's 
own json object
 + See CASSANDRA-13848 for more detail
 +
 +Upgrading
 +-
 +- Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact 
Tables
 +  have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 
3.0/3.11.
 +  Cassandra starting 4.0 will not start if flags indicate that the table 
is non-CQL.
 +  Syntax for creating compact tables is also deprecated.
 +- Support for legacy auth tables in the system_auth keyspace (users,
 +  permissions, credentials) and the migration code has been 

[jira] [Resolved] (CASSANDRA-12103) Cassandra is hang and cqlsh was not able to login with OperationTimeout error

[peng xiao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peng xiao resolved CASSANDRA-12103.
---
Resolution: Fixed

> Cassandra is hang and cqlsh was not able to login with OperationTimeout error
> -
>
> Key: CASSANDRA-12103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12103
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Local Write-Read Paths
> Environment: centos 6.5 cassandra 2.1.9
>Reporter: peng xiao
>Priority: Critical
>  Labels: proposed-wontfix
> Attachments: system.log.2016-06-28_1257.gz
>
>
> Hi,
> We have two DCs(DC1 and DC2) with DC1 3 nodes and DC2 9 nodes.
> And we experienced a Timeout error today,all applications connected to DC1 
> were hang and no response,even cqlsh was not able to log into any node in DC1.
> I restarted the 3 nodes in DC1,the problem was not resolved.
> Then we switched to DC2,then applications back to normal.
> Could you please help to take a look?
> Thanks
> many errors like below:
> ERROR [SharedPool-Worker-43] 2016-06-28 11:58:49,705 Message.java:538 - 
> Unexpected exception during request; channel = [id: 0x87e315d6, 
> /172.16.10.198:13604 => /172.16.11.13:9042]
> java.lang.RuntimeException: 
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
> received only 0 responses.
> at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:86) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.service.ClientState.login(ClientState.java:206) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.transport.messages.AuthResponse.execute(AuthResponse.java:82)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>  [apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>  [apache-cassandra-2.1.9.jar:2.1.9]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [apache-cassandra-2.1.9.jar:2.1.9]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-2.1.9.jar:2.1.9]
> at java.lang.Thread.run(Thread.java:744) [na:1.8.0]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12917) Increase error margin in SplitterTest

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-12917:

   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   4.0
   3.11.2
   Status: Resolved  (was: Ready to Commit)

ah thanks for the ping

committed the lazy approach since this I don't think this has failed for a long 
time (and bumping the margin should make it "never" fail...). 

> Increase error margin in SplitterTest
> -
>
> Key: CASSANDRA-12917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12917
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>  Labels: lhf
> Fix For: 3.11.2, 4.0
>
>
> SplitterTest is a randomized test - it generates random tokens and splits the 
> ranges in equal parts. Since it is random we sometimes get very big vnodes 
> right where we want a split and that makes the split unbalanced
> Bumping the error margin a bit will avoid these false positives.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/3] cassandra git commit: Increase error margin in SplitterTest

[marcuse]

Increase error margin in SplitterTest

Patch by marcuse; reviewed by Branimir Lambov for CASSANDRA-12917


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/88a41fb8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/88a41fb8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/88a41fb8

Branch: refs/heads/trunk
Commit: 88a41fb826d68c56e60f88ea4c00c1b92a653e28
Parents: 50e6e72
Author: Marcus Eriksson 
Authored: Wed Nov 16 16:32:04 2016 +0100
Committer: Marcus Eriksson 
Committed: Tue Dec 12 09:51:01 2017 +0100

--
 test/unit/org/apache/cassandra/dht/SplitterTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/88a41fb8/test/unit/org/apache/cassandra/dht/SplitterTest.java
--
diff --git a/test/unit/org/apache/cassandra/dht/SplitterTest.java 
b/test/unit/org/apache/cassandra/dht/SplitterTest.java
index 751a7d7..b39c09f 100644
--- a/test/unit/org/apache/cassandra/dht/SplitterTest.java
+++ b/test/unit/org/apache/cassandra/dht/SplitterTest.java
@@ -97,7 +97,7 @@ public class SplitterTest
 start = end;
 }
 // when we dont need to keep around full ranges, the difference is 
small between the partitions
-BigDecimal delta = splitIndividualRanges ? BigDecimal.valueOf(0.001) : 
BigDecimal.valueOf(0.2);
+BigDecimal delta = splitIndividualRanges ? BigDecimal.valueOf(0.001) : 
BigDecimal.valueOf(0.25);
 boolean allBalanced = true;
 for (BigInteger b : splits)
 {


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[marcuse]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8e95534f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8e95534f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8e95534f

Branch: refs/heads/trunk
Commit: 8e95534fb51aa76e5ef702b0015e30b27cf3d941
Parents: e18a49a 88a41fb
Author: Marcus Eriksson 
Authored: Tue Dec 12 09:52:06 2017 +0100
Committer: Marcus Eriksson 
Committed: Tue Dec 12 09:52:06 2017 +0100

--
 test/unit/org/apache/cassandra/dht/SplitterTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8e95534f/test/unit/org/apache/cassandra/dht/SplitterTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/3] cassandra git commit: Increase error margin in SplitterTest

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 50e6e721b -> 88a41fb82
  refs/heads/trunk e18a49a23 -> 8e95534fb


Increase error margin in SplitterTest

Patch by marcuse; reviewed by Branimir Lambov for CASSANDRA-12917


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/88a41fb8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/88a41fb8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/88a41fb8

Branch: refs/heads/cassandra-3.11
Commit: 88a41fb826d68c56e60f88ea4c00c1b92a653e28
Parents: 50e6e72
Author: Marcus Eriksson 
Authored: Wed Nov 16 16:32:04 2016 +0100
Committer: Marcus Eriksson 
Committed: Tue Dec 12 09:51:01 2017 +0100

--
 test/unit/org/apache/cassandra/dht/SplitterTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/88a41fb8/test/unit/org/apache/cassandra/dht/SplitterTest.java
--
diff --git a/test/unit/org/apache/cassandra/dht/SplitterTest.java 
b/test/unit/org/apache/cassandra/dht/SplitterTest.java
index 751a7d7..b39c09f 100644
--- a/test/unit/org/apache/cassandra/dht/SplitterTest.java
+++ b/test/unit/org/apache/cassandra/dht/SplitterTest.java
@@ -97,7 +97,7 @@ public class SplitterTest
 start = end;
 }
 // when we dont need to keep around full ranges, the difference is 
small between the partitions
-BigDecimal delta = splitIndividualRanges ? BigDecimal.valueOf(0.001) : 
BigDecimal.valueOf(0.2);
+BigDecimal delta = splitIndividualRanges ? BigDecimal.valueOf(0.001) : 
BigDecimal.valueOf(0.25);
 boolean allBalanced = true;
 for (BigInteger b : splits)
 {


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

[Rob Emery (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287278#comment-16287278
 ] 

Rob Emery commented on CASSANDRA-12978:
---

Hi Jeff,

That sounds perfectly acceptable.

Many Thanks,
Rob

> mx4j -> HTTP 500 -> ConcurrentModificationException
> ---
>
> Key: CASSANDRA-12978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 
> 16GB, RAID1 SSD Commit log, RAID10 15k HDD data
>Reporter: Rob Emery
>Priority: Critical
>  Labels: proposed-wontfix
> Fix For: 2.1.6
>
>
> We run some checks from our Monitoring software that rely on mx4j.
> The checks typically grab some xml via HTTP request and parse it. For 
> example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
> using:
> http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily
> The checks run each minute. Periodically they result in a "HTTP 500 internal 
> server error". The HTML body returned is empty.
> Experimentally we ran Cassandra in the foreground on one node and reproduced 
> the problem. this elicited the following stack trace:
> javax.management.RuntimeMBeanException: 
> java.util.ConcurrentModificationException
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
> at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
> at 
> mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
> Caused by: java.util.ConcurrentModificationException
> at 
> java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
> at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
> at 
> org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
> at 
> org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at 
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at 
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> ... 4 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13006) Disable automatic heap dumps on OOM error

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287277#comment-16287277
 ] 

Benjamin Lerer commented on CASSANDRA-13006:


[~jjirsa] Thanks for the reminder. 
I fixed the review comments and rerun CI on our internal instances. Tests looks 
good including the DTests ones (no new failing tests).


> Disable automatic heap dumps on OOM error
> -
>
> Key: CASSANDRA-13006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13006
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: anmols
>Assignee: Benjamin Lerer
>Priority: Minor
> Attachments: 13006-3.0.9.txt
>
>
> With CASSANDRA-9861, a change was added to enable collecting heap dumps by 
> default if the process encountered an OOM error. These heap dumps are stored 
> in the Apache Cassandra home directory unless configured otherwise (see 
> [Cassandra Support 
> Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps]
>  for this feature).
>  
> The creation and storage of heap dumps aides debugging and investigative 
> workflows, but is not be desirable for a production environment where these 
> heap dumps may occupy a large amount of disk space and require manual 
> intervention for cleanups. 
>  
> Managing heap dumps on out of memory errors and configuring the paths for 
> these heap dumps are available as JVM options in JVM. The current behavior 
> conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. 
>  
> A patch can be proposed here that would make the heap dump on OOM error honor 
> the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate 
> heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM 
> option.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org