[jira] [Commented] (CASSANDRA-12905) Retry acquire MV lock on failure instead of throwing WTE on streaming

[Benjamin Roth (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747566#comment-15747566
 ] 

Benjamin Roth commented on CASSANDRA-12905:
---

I like your naming changes, they make sense. But by making hint delivery async, 
you made it droppable again. I guess this is not intentional? You also do not 
reply on an exception. I am not familiar with request/response handling but I 
guess an exception (like WTE) will just drop the hint and let the hint-sender 
wait for a reply infinitely or until it times out?
The hint sender should be able to recover from an exception like 
re-transmitting the hints, right? I am not sure if this is the case here.

> Retry acquire MV lock on failure instead of throwing WTE on streaming
> -
>
> Key: CASSANDRA-12905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12905
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: centos 6.7 x86_64
>Reporter: Nir Zilka
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 3.10
>
>
> Hello,
> I performed two upgrades to the current cluster (currently 15 nodes, 1 DC, 
> private VLAN),
> first it was 2.2.5.1 and repair worked flawlessly,
> second upgrade was to 3.0.9 (with upgradesstables) and also repair worked 
> well,
> then i upgraded 2 weeks ago to 3.9 - and the repair problems started.
> there are several errors types from the system.log (different nodes) :
> - Sync failed between /xxx.xxx.xxx.xxx and /xxx.xxx.xxx.xxx
> - Streaming error occurred on session with peer xxx.xxx.xxx.xxx Operation 
> timed out - received only 0 responses
> - Remote peer xxx.xxx.xxx.xxx failed stream session
> - Session completed with the following error
> org.apache.cassandra.streaming.StreamException: Stream failed
> 
> i use 3.9 default configuration with the cluster settings adjustments (3 
> seeds, GossipingPropertyFileSnitch).
> streaming_socket_timeout_in_ms is the default (8640).
> i'm afraid from consistency problems while i'm not performing repair.
> Any ideas?
> Thanks,
> Nir.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12959) copy from csv import wrong values with udt having set when fields are not specified in correct order in csv

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747194#comment-15747194
 ] 

Stefania commented on CASSANDRA-12959:
--

Thanks for taking the review [~pauloricardomg]! Hopefully you haven't looked at 
the patch yet, and so it makes no difference, but I just modified it slightly 
so that we can handle missing UDT fields as well. The patch is still quite 
small and it applies without conflicts to all branches. CI is pending.

> copy from csv import wrong values with udt having set when fields are not 
> specified in correct order in csv
> ---
>
> Key: CASSANDRA-12959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12959
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Quentin Ambard
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
>  {quote}
> create KEYSPACE test WITH replication = \{ 'class': 'SimpleStrategy', 
> 'replication_factor': 1\};
> CREATE TYPE test.my_udt (
> first_field text,
> second_field frozen
> );
> CREATE TABLE test.test ( key text, value my_udt, PRIMARY KEY (key));
> {quote}
> The following works as expected : 
> {quote}
> INSERT INTO test.test (key , value ) VALUES ( 'key1', \{second_field: 
> \{'test1', 'test2'\}, first_field: 'first_field'\});
>  key  | value
> --+
>  key1 | \{first_field: 'first_field', second_field: \{'test1', 'test2'\}\}
> {quote}
> but when inserted using a .csv the result is wrong:
> {quote}"key1","\{second_field: \{'test1', 'test2'\}, first_field: 
> 'first_field'\}"
> COPY test.test FROM '~/test.csv';
>  key  | value
> --+--
>  key1 | \{first_field: '\{''test1'', ''test2''\}', second_field: 
> \{'irst_fiel'\}\}
> {quote}
> it works as expected if the keys are in order: 
> bq. "key1","\{first_field: 'first_field', second_field: \{'test1', 
> 'test2'\}\}")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12510) Disallow decommission when number of replicas will drop below configured RF

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747136#comment-15747136
 ] 

Paulo Motta commented on CASSANDRA-12510:
-

[resumable_decommission_test|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.X-12510-dtest/lastCompletedBuild/testReport/topology_test/TestTopology/resumable_decommission_test/]
 is still failing. Is there any particular reason why you made 
[this|https://github.com/apache/cassandra/compare/cassandra-3.X...pauloricardomg:3.X-12510#diff-b76a607445d53f18a98c9df14323c7ddR3823]
 change? It seems this will break resumable decommission (CASSANDRA-12008) 
because the local node (which is on LEAVING state) will be removed from the 
ring on {{tokenMetadata.cloneAfterAllSettled()}}.

We should probably skip altogether the checks of dropping below configured RF 
and no other normal nodes in the ring when the node is already decommisioning 
({{operationMode == Mode.LEAVING}}) to avoid breaking resumable decommission. 
Can you please make these changes? Thanks!

> Disallow decommission when number of replicas will drop below configured RF
> ---
>
> Key: CASSANDRA-12510
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12510
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
> Environment: C* version 3.3
>Reporter: Atin Sood
>Assignee: Kurt Greaves
>Priority: Minor
>  Labels: lhf
> Attachments: 12510-3.x.patch
>
>
> Steps to replicate :
> - Create a 3 node cluster in DC1 and create a keyspace test_keyspace with 
> table test_table with replication strategy NetworkTopologyStrategy , DC1=3 . 
> Populate some data into this table.
> - Add 5 more nodes to this cluster, but in DC2. Also do not alter the 
> keyspace to add the new DC2 to replication (this is intentional and the 
> reason why the bug shows up). So the desc keyspace should still list 
> NetworkTopologyStrategy with DC1=3 as RF
> - As expected, this will now be a 8 node cluster with 3 nodes in DC1 and 5 in 
> DC2
> - Now start decommissioning the nodes in DC1. Note that the decommission runs 
> fine on all the 3 nodes, but since the new nodes are in DC2 and the RF for 
> keyspace is restricted to DC1, the new 5 nodes won't get any data.
> - You will now end with the 5 node cluster which has no data from the 
> decommissioned 3 nodes and hence ending up in data loss
> I do understand that this problem could have been avoided if we perform an 
> alter stmt and add DC2 replication before adding the 5 nodes. But the fact 
> that decommission ran fine on the 3 nodes on DC1 without complaining that 
> there were no nodes to stream its data seems a little discomforting. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13042) The two cassandra nodes suddenly encounter hints each other and failed replaying.

[YheonHo.Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

YheonHo.Choi updated CASSANDRA-13042:
-
Description: 
Although there are no changes to cassandra, two node suddenly encounter hints 
and failed replaying.
Any commands like disablethrift, disablegossip can not solve the above problem 
and the only way was restart.
When we check the status of cluster, all nodes are looks UN but describecluster 
show unreachable each other.
Here's the state of the cassandra during the above problem occurred.

IP addresses in report anonymized:
cassandra version: 2.2.5
node 1 = 1.1.1.1
node 2 = 1.1.1.2
others = x.x.x.x

system.log
{code}
## result of nodetool status on 1.1.1.1
INFO  [HintedHandoff:1] 2016-11-24 06:15:07,969 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:15:09,969 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [HintedHandoff:2] 2016-11-24 06:25:09,736 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:2] 2016-11-24 06:25:11,738 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [MemtableFlushWriter:55270] 2016-11-24 06:25:12,625 
BigTableWriter.java:184 - Writing large partition 
system/hints:d640677d-f354-aa8c-be89-d2a1648c24b2 (109029803 bytes)
WARN  [CompactionExecutor:37908] 2016-11-24 06:35:23,682 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (250651758 bytes)
INFO  [HintedHandoff:1] 2016-11-24 06:35:23,727 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:35:25,728 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [CompactionExecutor:37909] 2016-11-24 06:45:53,615 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (340801514 bytes)
INFO  [HintedHandoff:2] 2016-11-24 06:45:53,718 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:2] 2016-11-24 06:45:55,719 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [CompactionExecutor:37912] 2016-11-24 06:56:20,884 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (472465093 bytes)
INFO  [HintedHandoff:1] 2016-11-24 06:56:20,966 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:56:22,967 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [CompactionExecutor:37911] 2016-11-24 07:07:12,568 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (577392172 bytes)
INFO  [HintedHandoff:2] 2016-11-24 07:07:12,643 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:2] 2016-11-24 07:07:14,643 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [IndexSummaryManager:1] 2016-11-24 07:09:15,929 
IndexSummaryRedistribution.java:74 - Redistributing index summaries

## result of nodetool status on 1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:11:37,300 HintedHandOffManager.java:367 - 
Started hinted handoff for host: b79124e9-394c-4400-a8e7-a0c94aec6878 with IP: 
/1.1.1.1
INFO  [HintedHandoff:1] 2016-11-24 06:11:39,301 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.1; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [HintedHandoff:2] 2016-11-24 06:22:17,946 HintedHandOffManager.java:367 - 
Started hinted handoff for host: b79124e9-394c-4400-a8e7-a0c94aec6878 with IP: 
/1.1.1.1
INFO  [HintedHandoff:2] 2016-11-24 06:22:19,948 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.1; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [IndexSummaryManager:1] 2016-11-24 06:27:00,177 
IndexSummaryRedistribution.java:74 - Redistributing index summaries
WARN  [CompactionExecutor:2315] 2016-11-24 06:32:28,159 BigTableWriter.java:184 
- 

[jira] [Created] (CASSANDRA-13042) The two cassandra nodes suddenly encounter hints each other and failed replaying.

[YheonHo.Choi (JIRA)]

YheonHo.Choi created CASSANDRA-13042:


 Summary: The two cassandra nodes suddenly encounter hints each 
other and failed replaying.
 Key: CASSANDRA-13042
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13042
 Project: Cassandra
  Issue Type: Bug
Reporter: YheonHo.Choi
Priority: Minor


Although there are no changes to cassandra, two node suddenly encounter hints 
and failed replaying.
Any commands like disablethrift, disablegossip can not solve the above problem 
and the only way was restart.
When we check the status of cluster, all nodes are looks UN but describecluster 
show unreachable each other.
Here's the state of the cassandra during the above problem occurred.

IP addresses in report anonymized:
cassandra version: 2.2.5
node 1 = 1.1.1.1
node 2 = 1.1.1.2
others = x.x.x.x

system.log
{code}
## result of nodetool status on 1.1.1.1
INFO  [HintedHandoff:1] 2016-11-24 06:15:07,969 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:15:09,969 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [HintedHandoff:2] 2016-11-24 06:25:09,736 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:2] 2016-11-24 06:25:11,738 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [MemtableFlushWriter:55270] 2016-11-24 06:25:12,625 
BigTableWriter.java:184 - Writing large partition 
system/hints:d640677d-f354-aa8c-be89-d2a1648c24b2 (109029803 bytes)
WARN  [CompactionExecutor:37908] 2016-11-24 06:35:23,682 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (250651758 bytes)
INFO  [HintedHandoff:1] 2016-11-24 06:35:23,727 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:35:25,728 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [CompactionExecutor:37909] 2016-11-24 06:45:53,615 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (340801514 bytes)
INFO  [HintedHandoff:2] 2016-11-24 06:45:53,718 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:2] 2016-11-24 06:45:55,719 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [CompactionExecutor:37912] 2016-11-24 06:56:20,884 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (472465093 bytes)
INFO  [HintedHandoff:1] 2016-11-24 06:56:20,966 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:56:22,967 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
WARN  [CompactionExecutor:37911] 2016-11-24 07:07:12,568 
BigTableWriter.java:184 - Writing large partition 
system/hints:8caa54f3-7d67-40d6-b224-8c64a1d289be (577392172 bytes)
INFO  [HintedHandoff:2] 2016-11-24 07:07:12,643 HintedHandOffManager.java:367 - 
Started hinted handoff for host: 8caa54f3-7d67-40d6-b224-8c64a1d289be with IP: 
/1.1.1.2
INFO  [HintedHandoff:2] 2016-11-24 07:07:14,643 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.2; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [IndexSummaryManager:1] 2016-11-24 07:09:15,929 
IndexSummaryRedistribution.java:74 - Redistributing index summaries

## result of nodetool status on 1.1.1.2
INFO  [HintedHandoff:1] 2016-11-24 06:11:37,300 HintedHandOffManager.java:367 - 
Started hinted handoff for host: b79124e9-394c-4400-a8e7-a0c94aec6878 with IP: 
/1.1.1.1
INFO  [HintedHandoff:1] 2016-11-24 06:11:39,301 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.1; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  [HintedHandoff:2] 2016-11-24 06:22:17,946 HintedHandOffManager.java:367 - 
Started hinted handoff for host: b79124e9-394c-4400-a8e7-a0c94aec6878 with IP: 
/1.1.1.1
INFO  [HintedHandoff:2] 2016-11-24 06:22:19,948 HintedHandOffManager.java:486 - 
Failed replaying hints to /1.1.1.1; aborting (0 delivered), error : Operation 
timed out - received only 0 responses.
INFO  

[jira] [Created] (CASSANDRA-13041) Do not allow removal of a DC from system_auth replication settings if the DC has active Cassandra instances

[Nachiket Patil (JIRA)]

Nachiket Patil created CASSANDRA-13041:
--

 Summary: Do not allow removal of a DC from system_auth replication 
settings if the DC has active Cassandra instances
 Key: CASSANDRA-13041
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13041
 Project: Cassandra
  Issue Type: Improvement
  Components: Distributed Metadata
Reporter: Nachiket Patil
Assignee: Nachiket Patil
Priority: Minor
 Fix For: 4.x


I don’t believe it is ever correct to remove a DC from the system_auth 
replication settings while there are nodes up in that DC. Cassandra should not 
allow this change if there are hosts which are currently members of the cluster 
in that DC, as any request which is routed to these hosts will meet an 
unavailable. Also dropping the keyspace system_auth should not be allowed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12905) Retry acquire MV lock on failure instead of throwing WTE on streaming

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746889#comment-15746889
 ] 

Paulo Motta commented on CASSANDRA-12905:
-

Updated 3.0 and 3.X patches with following changes:
* Rename/Invert {{dontTimeout}} flag from {{keyspace.apply}} to {{isDroppable}}
* Make hint delivery async (deferred) so it does not block on mutation stage on 
failure to acquire MV lock
* Rename all {{Keyspace.apply}} methods that return {{CompletableFuture}} to 
{{applyFuture}} so it is consistent with {{Mutation.applyFuture}} and more 
clear to avoid people calling it and expect to be blocking, and also renamed 
previously {{Keyspace.applyBlocking}} to {{Keyspace.apply}} since this retains 
the original nomenclature before CASSANDRA-10779 when it does not return a 
{{CompletableFuture}}, to avoid confusing users (given there is an 
{{applyFuture}} method).
* On 3.x patch, write mutations to commit log when cdc is enabled.

[~brstgt] can you please double check and validate these changes?

Updated patch and resubmitted CI results below:
||3.0||3.X||
|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-12905]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.X...pauloricardomg:3.X-12905]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-12905-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.X-12905-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-12905-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.X-12905-dtest/lastCompletedBuild/testReport/]|

> Retry acquire MV lock on failure instead of throwing WTE on streaming
> -
>
> Key: CASSANDRA-12905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12905
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: centos 6.7 x86_64
>Reporter: Nir Zilka
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 3.10
>
>
> Hello,
> I performed two upgrades to the current cluster (currently 15 nodes, 1 DC, 
> private VLAN),
> first it was 2.2.5.1 and repair worked flawlessly,
> second upgrade was to 3.0.9 (with upgradesstables) and also repair worked 
> well,
> then i upgraded 2 weeks ago to 3.9 - and the repair problems started.
> there are several errors types from the system.log (different nodes) :
> - Sync failed between /xxx.xxx.xxx.xxx and /xxx.xxx.xxx.xxx
> - Streaming error occurred on session with peer xxx.xxx.xxx.xxx Operation 
> timed out - received only 0 responses
> - Remote peer xxx.xxx.xxx.xxx failed stream session
> - Session completed with the following error
> org.apache.cassandra.streaming.StreamException: Stream failed
> 
> i use 3.9 default configuration with the cluster settings adjustments (3 
> seeds, GossipingPropertyFileSnitch).
> streaming_socket_timeout_in_ms is the default (8640).
> i'm afraid from consistency problems while i'm not performing repair.
> Any ideas?
> Thanks,
> Nir.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12905) Retry acquire MV lock on failure instead of throwing WTE on streaming

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12905:

Status: Patch Available  (was: Awaiting Feedback)

> Retry acquire MV lock on failure instead of throwing WTE on streaming
> -
>
> Key: CASSANDRA-12905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12905
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: centos 6.7 x86_64
>Reporter: Nir Zilka
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 3.10
>
>
> Hello,
> I performed two upgrades to the current cluster (currently 15 nodes, 1 DC, 
> private VLAN),
> first it was 2.2.5.1 and repair worked flawlessly,
> second upgrade was to 3.0.9 (with upgradesstables) and also repair worked 
> well,
> then i upgraded 2 weeks ago to 3.9 - and the repair problems started.
> there are several errors types from the system.log (different nodes) :
> - Sync failed between /xxx.xxx.xxx.xxx and /xxx.xxx.xxx.xxx
> - Streaming error occurred on session with peer xxx.xxx.xxx.xxx Operation 
> timed out - received only 0 responses
> - Remote peer xxx.xxx.xxx.xxx failed stream session
> - Session completed with the following error
> org.apache.cassandra.streaming.StreamException: Stream failed
> 
> i use 3.9 default configuration with the cluster settings adjustments (3 
> seeds, GossipingPropertyFileSnitch).
> streaming_socket_timeout_in_ms is the default (8640).
> i'm afraid from consistency problems while i'm not performing repair.
> Any ideas?
> Thanks,
> Nir.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746857#comment-15746857
 ] 

Jason Brown commented on CASSANDRA-13040:
-

[~iksaif] so, are you saying this ticket has introduced a correctness bug or a 
performance regression, or that you feel CASSANDRA-13038 (which I haven't 
looked at yet) will additionally improve upon this patch?

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 3.0.11, 3.10, 3.12, 4.0
>
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: remove dead allocation

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1a0c0d375 -> 0c56d6c22


remove dead allocation


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0c56d6c2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0c56d6c2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0c56d6c2

Branch: refs/heads/trunk
Commit: 0c56d6c2230bafb800360a0fc1e09a112b2862aa
Parents: 1a0c0d3
Author: Dave Brosius 
Authored: Tue Dec 13 20:06:29 2016 -0500
Committer: Dave Brosius 
Committed: Tue Dec 13 20:06:29 2016 -0500

--
 tools/stress/src/org/apache/cassandra/stress/StressProfile.java | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0c56d6c2/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 95df986..e15e0ba 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -364,7 +364,6 @@ public class StressProfile implements Serializable
 JavaDriverClient jclient = settings.getJavaDriverClient();
 
 Map stmts = new HashMap<>();
-Map tids = new HashMap<>();
 Map args = new HashMap<>();
 for (Map.Entry e : 
queries.entrySet())
 {

cassandra git commit: don't declare throwing exceptions that aren't

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 402f8e006 -> 1a0c0d375


don't declare throwing exceptions that aren't


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a0c0d37
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a0c0d37
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a0c0d37

Branch: refs/heads/trunk
Commit: 1a0c0d375bffcd757a5ccec20190114798b31fba
Parents: 402f8e0
Author: Dave Brosius 
Authored: Tue Dec 13 19:59:28 2016 -0500
Committer: Dave Brosius 
Committed: Tue Dec 13 19:59:52 2016 -0500

--
 .../org/apache/cassandra/db/compaction/CompactionManager.java| 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a0c0d37/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 3ad7b38..baa82bc 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -364,7 +364,7 @@ public class CompactionManager implements 
CompactionManagerMBean
 }
 
 @Override
-public void execute(LifecycleTransaction input) throws IOException
+public void execute(LifecycleTransaction input)
 {
 scrubOne(cfs, input, skipCorrupted, checkData);
 }
@@ -953,7 +953,7 @@ public class CompactionManager implements 
CompactionManagerMBean
 }
 }
 
-private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction 
modifier, boolean skipCorrupted, boolean checkData) throws IOException
+private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction 
modifier, boolean skipCorrupted, boolean checkData)
 {
 CompactionInfo.Holder scrubInfo = null;
 

[jira] [Updated] (CASSANDRA-8616) sstable tools may result in commit log segments be written

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-8616:
--
   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 2.2.x)
   (was: 3.x)
   3.10
   3.0.11
   2.2.9
Reproduced In: 2.1.3, 2.0.10  (was: 2.0.10, 2.1.3)
   Status: Resolved  (was: Patch Available)

Thanks, committed as {{66f1aaf88d3cde5c52b13d71d3326da5eda16fb1}}.

> sstable tools may result in commit log segments be written
> --
>
> Key: CASSANDRA-8616
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8616
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Yuki Morishita
> Fix For: 2.2.9, 3.0.11, 3.10
>
> Attachments: 8161-2.0.txt
>
>
> There was a report of sstable2json causing commitlog segments to be written 
> out when run.  I haven't attempted to reproduce this yet, so that's all I 
> know for now.  Since sstable2json loads the conf and schema, I'm thinking 
> that it may inadvertently be triggering the commitlog code.
> sstablescrub, sstableverify, and other sstable tools have the same issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[08/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --cc src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 003b624,000..a6ed3ba
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@@ -1,1112 -1,0 +1,1114 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.index;
 +
 +import java.lang.reflect.Constructor;
 +import java.util.*;
 +import java.util.concurrent.*;
 +import java.util.function.Function;
 +import java.util.stream.Collectors;
 +import java.util.stream.Stream;
 +
 +import com.google.common.base.Joiner;
 +import com.google.common.base.Strings;
 +import com.google.common.collect.ImmutableSet;
 +import com.google.common.collect.Iterables;
 +import com.google.common.collect.Maps;
 +import com.google.common.collect.Sets;
 +import com.google.common.primitives.Longs;
 +import com.google.common.util.concurrent.Futures;
 +import com.google.common.util.concurrent.MoreExecutors;
 +import org.apache.commons.lang3.StringUtils;
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor;
 +import org.apache.cassandra.concurrent.NamedThreadFactory;
 +import org.apache.cassandra.concurrent.StageManager;
 +import org.apache.cassandra.config.ColumnDefinition;
++import org.apache.cassandra.config.DatabaseDescriptor;
 +import org.apache.cassandra.cql3.statements.IndexTarget;
 +import org.apache.cassandra.db.*;
 +import org.apache.cassandra.db.compaction.CompactionManager;
 +import org.apache.cassandra.db.filter.RowFilter;
 +import org.apache.cassandra.db.lifecycle.SSTableSet;
 +import org.apache.cassandra.db.lifecycle.View;
 +import org.apache.cassandra.db.partitions.PartitionIterators;
 +import org.apache.cassandra.db.partitions.PartitionUpdate;
 +import org.apache.cassandra.db.rows.*;
 +import org.apache.cassandra.exceptions.InvalidRequestException;
 +import org.apache.cassandra.index.internal.CassandraIndex;
 +import org.apache.cassandra.index.transactions.*;
 +import org.apache.cassandra.io.sstable.ReducingKeyIterator;
 +import org.apache.cassandra.io.sstable.format.SSTableReader;
 +import org.apache.cassandra.schema.IndexMetadata;
 +import org.apache.cassandra.schema.Indexes;
 +import org.apache.cassandra.service.pager.SinglePartitionPager;
 +import org.apache.cassandra.tracing.Tracing;
 +import org.apache.cassandra.transport.Server;
 +import org.apache.cassandra.utils.FBUtilities;
 +import org.apache.cassandra.utils.concurrent.OpOrder;
 +import org.apache.cassandra.utils.concurrent.Refs;
 +
 +/**
 + * Handles the core maintenance functionality associated with indexes: 
adding/removing them to or from
 + * a table, (re)building during bootstrap or other streaming operations, 
flushing, reloading metadata
 + * and so on.
 + *
 + * The Index interface defines a number of methods which return Callable. 
These are primarily the
 + * management tasks for an index implementation. Most of them are currently 
executed in a blocking
 + * fashion via submission to SIM's blockingExecutor. This provides the 
desired behaviour in pretty
 + * much all cases, as tasks like flushing an index needs to be executed 
synchronously to avoid potentially
 + * deadlocking on the FlushWriter or PostFlusher. Several of these 
Callable returning methods on Index could
 + * then be defined with as void and called directly from SIM (rather than 
being run via the executor service).
 + * Separating the task defintion from execution gives us greater flexibility 
though, so that in future, for example,
 + * if the flush process allows it we leave open the possibility of executing 
more of these tasks asynchronously.
 + *
 + * The primary exception to the above is the Callable returned from 
Index#addIndexedColumn. This may
 + * involve a significant effort, building a new index over any existing data. 
We perform this task asynchronously;
 + * as it is called as part of a schema 

[17/19] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[yukim]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9ead482
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9ead482
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9ead482

Branch: refs/heads/cassandra-3.X
Commit: c9ead4826a1dfb246b4ef6c916106bb13fcdbdc5
Parents: ddefb1d 66e2145
Author: Yuki Morishita 
Authored: Tue Dec 13 15:59:23 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:59:23 2016 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 ++-
 .../cassandra/db/PartitionRangeReadCommand.java |  4 +-
 .../cassandra/db/compaction/CompactionTask.java | 81 ++--
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 +-
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 +-
 .../cassandra/db/lifecycle/TrackerTest.java |  6 +-
 .../org/apache/cassandra/tools/ToolsTester.java |  3 -
 10 files changed, 85 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9ead482/CHANGES.txt
--

[03/19] cassandra git commit: Temporarily fix bug that creates commit log when running offline tools

[yukim]

Temporarily fix bug that creates commit log when running offline tools

patch by yukim; reviewed by thobbs for CASSANDRA-8616


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66f1aaf8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66f1aaf8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66f1aaf8

Branch: refs/heads/trunk
Commit: 66f1aaf88d3cde5c52b13d71d3326da5eda16fb1
Parents: fb29400
Author: Yuki Morishita 
Authored: Thu Feb 11 19:06:27 2016 -0600
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:51:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/compaction/CompactionTask.java | 22 +++-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 36 
 .../io/sstable/format/SSTableReader.java|  8 +++--
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../io/sstable/CQLSSTableWriterLongTest.java|  2 ++
 .../unit/org/apache/cassandra/SchemaLoader.java |  1 +
 .../config/DatabaseDescriptorTest.java  |  6 
 .../apache/cassandra/db/CounterCellTest.java|  3 ++
 .../org/apache/cassandra/db/NativeCellTest.java |  3 ++
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../service/StorageServiceServerTest.java   |  1 +
 25 files changed, 118 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f02350d..8cff097 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
  * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
  * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
  * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3fb79b..35debd0 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -102,6 +102,18 @@ public class DatabaseDescriptor
 private static Comparator localComparator;
 private static boolean hasLoggedConfig;
 
+private static boolean daemonInitialized;
+
+public static boolean isDaemonInitialized()
+{
+return daemonInitialized;
+}
+
+public static void setDaemonInitialized()
+{
+daemonInitialized = true;
+}
+
 public static void forceStaticInitialization() {}
 static
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index c6b69dc..4bc46d0 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -388,9 +388,13 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 logger.info("Initializing {}.{}", keyspace.getName(), name);
 
-// scan for sstables corresponding to this cf and load them
-data = new Tracker(this, loadSSTables);
+// Create Memtable only on online
+Memtable initialMemtable = null;
+if (DatabaseDescriptor.isDaemonInitialized())
+initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
+data = new 

[05/19] cassandra git commit: Temporarily fix bug that creates commit log when running offline tools

[yukim]

Temporarily fix bug that creates commit log when running offline tools

patch by yukim; reviewed by thobbs for CASSANDRA-8616


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66f1aaf8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66f1aaf8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66f1aaf8

Branch: refs/heads/cassandra-3.X
Commit: 66f1aaf88d3cde5c52b13d71d3326da5eda16fb1
Parents: fb29400
Author: Yuki Morishita 
Authored: Thu Feb 11 19:06:27 2016 -0600
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:51:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/compaction/CompactionTask.java | 22 +++-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 36 
 .../io/sstable/format/SSTableReader.java|  8 +++--
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../io/sstable/CQLSSTableWriterLongTest.java|  2 ++
 .../unit/org/apache/cassandra/SchemaLoader.java |  1 +
 .../config/DatabaseDescriptorTest.java  |  6 
 .../apache/cassandra/db/CounterCellTest.java|  3 ++
 .../org/apache/cassandra/db/NativeCellTest.java |  3 ++
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../service/StorageServiceServerTest.java   |  1 +
 25 files changed, 118 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f02350d..8cff097 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
  * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
  * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
  * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3fb79b..35debd0 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -102,6 +102,18 @@ public class DatabaseDescriptor
 private static Comparator localComparator;
 private static boolean hasLoggedConfig;
 
+private static boolean daemonInitialized;
+
+public static boolean isDaemonInitialized()
+{
+return daemonInitialized;
+}
+
+public static void setDaemonInitialized()
+{
+daemonInitialized = true;
+}
+
 public static void forceStaticInitialization() {}
 static
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index c6b69dc..4bc46d0 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -388,9 +388,13 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 logger.info("Initializing {}.{}", keyspace.getName(), name);
 
-// scan for sstables corresponding to this cf and load them
-data = new Tracker(this, loadSSTables);
+// Create Memtable only on online
+Memtable initialMemtable = null;
+if (DatabaseDescriptor.isDaemonInitialized())
+initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
+data = new 

[02/19] cassandra git commit: Temporarily fix bug that creates commit log when running offline tools

[yukim]

Temporarily fix bug that creates commit log when running offline tools

patch by yukim; reviewed by thobbs for CASSANDRA-8616


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66f1aaf8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66f1aaf8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66f1aaf8

Branch: refs/heads/cassandra-3.0
Commit: 66f1aaf88d3cde5c52b13d71d3326da5eda16fb1
Parents: fb29400
Author: Yuki Morishita 
Authored: Thu Feb 11 19:06:27 2016 -0600
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:51:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/compaction/CompactionTask.java | 22 +++-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 36 
 .../io/sstable/format/SSTableReader.java|  8 +++--
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../io/sstable/CQLSSTableWriterLongTest.java|  2 ++
 .../unit/org/apache/cassandra/SchemaLoader.java |  1 +
 .../config/DatabaseDescriptorTest.java  |  6 
 .../apache/cassandra/db/CounterCellTest.java|  3 ++
 .../org/apache/cassandra/db/NativeCellTest.java |  3 ++
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../service/StorageServiceServerTest.java   |  1 +
 25 files changed, 118 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f02350d..8cff097 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
  * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
  * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
  * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3fb79b..35debd0 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -102,6 +102,18 @@ public class DatabaseDescriptor
 private static Comparator localComparator;
 private static boolean hasLoggedConfig;
 
+private static boolean daemonInitialized;
+
+public static boolean isDaemonInitialized()
+{
+return daemonInitialized;
+}
+
+public static void setDaemonInitialized()
+{
+daemonInitialized = true;
+}
+
 public static void forceStaticInitialization() {}
 static
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index c6b69dc..4bc46d0 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -388,9 +388,13 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 logger.info("Initializing {}.{}", keyspace.getName(), name);
 
-// scan for sstables corresponding to this cf and load them
-data = new Tracker(this, loadSSTables);
+// Create Memtable only on online
+Memtable initialMemtable = null;
+if (DatabaseDescriptor.isDaemonInitialized())
+initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
+data = new 

[06/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --cc src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 003b624,000..a6ed3ba
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@@ -1,1112 -1,0 +1,1114 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.index;
 +
 +import java.lang.reflect.Constructor;
 +import java.util.*;
 +import java.util.concurrent.*;
 +import java.util.function.Function;
 +import java.util.stream.Collectors;
 +import java.util.stream.Stream;
 +
 +import com.google.common.base.Joiner;
 +import com.google.common.base.Strings;
 +import com.google.common.collect.ImmutableSet;
 +import com.google.common.collect.Iterables;
 +import com.google.common.collect.Maps;
 +import com.google.common.collect.Sets;
 +import com.google.common.primitives.Longs;
 +import com.google.common.util.concurrent.Futures;
 +import com.google.common.util.concurrent.MoreExecutors;
 +import org.apache.commons.lang3.StringUtils;
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor;
 +import org.apache.cassandra.concurrent.NamedThreadFactory;
 +import org.apache.cassandra.concurrent.StageManager;
 +import org.apache.cassandra.config.ColumnDefinition;
++import org.apache.cassandra.config.DatabaseDescriptor;
 +import org.apache.cassandra.cql3.statements.IndexTarget;
 +import org.apache.cassandra.db.*;
 +import org.apache.cassandra.db.compaction.CompactionManager;
 +import org.apache.cassandra.db.filter.RowFilter;
 +import org.apache.cassandra.db.lifecycle.SSTableSet;
 +import org.apache.cassandra.db.lifecycle.View;
 +import org.apache.cassandra.db.partitions.PartitionIterators;
 +import org.apache.cassandra.db.partitions.PartitionUpdate;
 +import org.apache.cassandra.db.rows.*;
 +import org.apache.cassandra.exceptions.InvalidRequestException;
 +import org.apache.cassandra.index.internal.CassandraIndex;
 +import org.apache.cassandra.index.transactions.*;
 +import org.apache.cassandra.io.sstable.ReducingKeyIterator;
 +import org.apache.cassandra.io.sstable.format.SSTableReader;
 +import org.apache.cassandra.schema.IndexMetadata;
 +import org.apache.cassandra.schema.Indexes;
 +import org.apache.cassandra.service.pager.SinglePartitionPager;
 +import org.apache.cassandra.tracing.Tracing;
 +import org.apache.cassandra.transport.Server;
 +import org.apache.cassandra.utils.FBUtilities;
 +import org.apache.cassandra.utils.concurrent.OpOrder;
 +import org.apache.cassandra.utils.concurrent.Refs;
 +
 +/**
 + * Handles the core maintenance functionality associated with indexes: 
adding/removing them to or from
 + * a table, (re)building during bootstrap or other streaming operations, 
flushing, reloading metadata
 + * and so on.
 + *
 + * The Index interface defines a number of methods which return Callable. 
These are primarily the
 + * management tasks for an index implementation. Most of them are currently 
executed in a blocking
 + * fashion via submission to SIM's blockingExecutor. This provides the 
desired behaviour in pretty
 + * much all cases, as tasks like flushing an index needs to be executed 
synchronously to avoid potentially
 + * deadlocking on the FlushWriter or PostFlusher. Several of these 
Callable returning methods on Index could
 + * then be defined with as void and called directly from SIM (rather than 
being run via the executor service).
 + * Separating the task defintion from execution gives us greater flexibility 
though, so that in future, for example,
 + * if the flush process allows it we leave open the possibility of executing 
more of these tasks asynchronously.
 + *
 + * The primary exception to the above is the Callable returned from 
Index#addIndexedColumn. This may
 + * involve a significant effort, building a new index over any existing data. 
We perform this task asynchronously;
 + * as it is called as part of a schema 

[13/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0fe82be8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0fe82be8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0fe82be8

Branch: refs/heads/cassandra-3.X
Commit: 0fe82be83cceceb12172d63913388678253413bc
Parents: e9b7a0f 66f1aaf
Author: Yuki Morishita 
Authored: Tue Dec 13 15:55:34 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:55:34 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/PartitionRangeReadCommand.java |  3 +-
 .../cassandra/db/compaction/CompactionTask.java | 18 +++
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 ++-
 .../io/sstable/format/SSTableReader.java|  2 +-
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../config/DatabaseDescriptorTest.java  |  6 
 .../org/apache/cassandra/cql3/CQLTester.java|  1 +
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../metrics/HintedHandOffMetricsTest.java   |  7 
 .../service/StorageServiceServerTest.java   |  1 +
 .../concurrent/AbstractTransactionalTest.java   |  7 
 27 files changed, 124 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/CHANGES.txt
--
diff --cc CHANGES.txt
index 5621c93,8cff097..145afb9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.9
 +3.0.11
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * CQL often queries static columns unnecessarily (CASSANDRA-12768)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
   * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
   * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
   * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 71e1653,4bc46d0..39ed804
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -388,13 -388,17 +388,17 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+   

[12/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --cc src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 003b624,000..a6ed3ba
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@@ -1,1112 -1,0 +1,1114 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.index;
 +
 +import java.lang.reflect.Constructor;
 +import java.util.*;
 +import java.util.concurrent.*;
 +import java.util.function.Function;
 +import java.util.stream.Collectors;
 +import java.util.stream.Stream;
 +
 +import com.google.common.base.Joiner;
 +import com.google.common.base.Strings;
 +import com.google.common.collect.ImmutableSet;
 +import com.google.common.collect.Iterables;
 +import com.google.common.collect.Maps;
 +import com.google.common.collect.Sets;
 +import com.google.common.primitives.Longs;
 +import com.google.common.util.concurrent.Futures;
 +import com.google.common.util.concurrent.MoreExecutors;
 +import org.apache.commons.lang3.StringUtils;
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor;
 +import org.apache.cassandra.concurrent.NamedThreadFactory;
 +import org.apache.cassandra.concurrent.StageManager;
 +import org.apache.cassandra.config.ColumnDefinition;
++import org.apache.cassandra.config.DatabaseDescriptor;
 +import org.apache.cassandra.cql3.statements.IndexTarget;
 +import org.apache.cassandra.db.*;
 +import org.apache.cassandra.db.compaction.CompactionManager;
 +import org.apache.cassandra.db.filter.RowFilter;
 +import org.apache.cassandra.db.lifecycle.SSTableSet;
 +import org.apache.cassandra.db.lifecycle.View;
 +import org.apache.cassandra.db.partitions.PartitionIterators;
 +import org.apache.cassandra.db.partitions.PartitionUpdate;
 +import org.apache.cassandra.db.rows.*;
 +import org.apache.cassandra.exceptions.InvalidRequestException;
 +import org.apache.cassandra.index.internal.CassandraIndex;
 +import org.apache.cassandra.index.transactions.*;
 +import org.apache.cassandra.io.sstable.ReducingKeyIterator;
 +import org.apache.cassandra.io.sstable.format.SSTableReader;
 +import org.apache.cassandra.schema.IndexMetadata;
 +import org.apache.cassandra.schema.Indexes;
 +import org.apache.cassandra.service.pager.SinglePartitionPager;
 +import org.apache.cassandra.tracing.Tracing;
 +import org.apache.cassandra.transport.Server;
 +import org.apache.cassandra.utils.FBUtilities;
 +import org.apache.cassandra.utils.concurrent.OpOrder;
 +import org.apache.cassandra.utils.concurrent.Refs;
 +
 +/**
 + * Handles the core maintenance functionality associated with indexes: 
adding/removing them to or from
 + * a table, (re)building during bootstrap or other streaming operations, 
flushing, reloading metadata
 + * and so on.
 + *
 + * The Index interface defines a number of methods which return Callable. 
These are primarily the
 + * management tasks for an index implementation. Most of them are currently 
executed in a blocking
 + * fashion via submission to SIM's blockingExecutor. This provides the 
desired behaviour in pretty
 + * much all cases, as tasks like flushing an index needs to be executed 
synchronously to avoid potentially
 + * deadlocking on the FlushWriter or PostFlusher. Several of these 
Callable returning methods on Index could
 + * then be defined with as void and called directly from SIM (rather than 
being run via the executor service).
 + * Separating the task defintion from execution gives us greater flexibility 
though, so that in future, for example,
 + * if the flush process allows it we leave open the possibility of executing 
more of these tasks asynchronously.
 + *
 + * The primary exception to the above is the Callable returned from 
Index#addIndexedColumn. This may
 + * involve a significant effort, building a new index over any existing data. 
We perform this task asynchronously;
 + * as it is called as part of a schema 

[19/19] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[yukim]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/402f8e00
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/402f8e00
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/402f8e00

Branch: refs/heads/trunk
Commit: 402f8e00606a54dcac46c60463358501a6a2e686
Parents: cd5aacc c9ead48
Author: Yuki Morishita 
Authored: Tue Dec 13 16:27:05 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 16:27:05 2016 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 ++-
 .../cassandra/db/PartitionRangeReadCommand.java |  4 +-
 .../cassandra/db/compaction/CompactionTask.java | 81 ++--
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 +-
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 +-
 .../cassandra/db/lifecycle/TrackerTest.java |  6 +-
 .../org/apache/cassandra/tools/ToolsTester.java |  3 -
 10 files changed, 85 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/402f8e00/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/402f8e00/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/402f8e00/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
--
diff --cc src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
index 9477aeb,045fc26..df4d63c
--- a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
+++ b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
@@@ -219,7 -221,9 +219,9 @@@ public class PartitionRangeReadCommand 
  if (!sstable.isRepaired())
  oldestUnrepairedTombstone = 
Math.min(oldestUnrepairedTombstone, sstable.getMinLocalDeletionTime());
  }
- return 
checkCacheFilter(UnfilteredPartitionIterators.mergeLazily(iterators, 
nowInSec()), cfs);
+ // iterators can be empty for offline tools
 -return iterators.isEmpty() ? 
EmptyIterators.unfilteredPartition(metadata(), isForThrift())
++return iterators.isEmpty() ? 
EmptyIterators.unfilteredPartition(metadata())
+: 
checkCacheFilter(UnfilteredPartitionIterators.mergeLazily(iterators, 
nowInSec()), cfs);
  }
  catch (RuntimeException | Error e)
  {

[11/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0fe82be8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0fe82be8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0fe82be8

Branch: refs/heads/cassandra-3.11
Commit: 0fe82be83cceceb12172d63913388678253413bc
Parents: e9b7a0f 66f1aaf
Author: Yuki Morishita 
Authored: Tue Dec 13 15:55:34 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:55:34 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/PartitionRangeReadCommand.java |  3 +-
 .../cassandra/db/compaction/CompactionTask.java | 18 +++
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 ++-
 .../io/sstable/format/SSTableReader.java|  2 +-
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../config/DatabaseDescriptorTest.java  |  6 
 .../org/apache/cassandra/cql3/CQLTester.java|  1 +
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../metrics/HintedHandOffMetricsTest.java   |  7 
 .../service/StorageServiceServerTest.java   |  1 +
 .../concurrent/AbstractTransactionalTest.java   |  7 
 27 files changed, 124 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/CHANGES.txt
--
diff --cc CHANGES.txt
index 5621c93,8cff097..145afb9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.9
 +3.0.11
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * CQL often queries static columns unnecessarily (CASSANDRA-12768)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
   * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
   * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
   * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 71e1653,4bc46d0..39ed804
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -388,13 -388,17 +388,17 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+  

[18/19] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[yukim]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9ead482
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9ead482
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9ead482

Branch: refs/heads/trunk
Commit: c9ead4826a1dfb246b4ef6c916106bb13fcdbdc5
Parents: ddefb1d 66e2145
Author: Yuki Morishita 
Authored: Tue Dec 13 15:59:23 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:59:23 2016 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 ++-
 .../cassandra/db/PartitionRangeReadCommand.java |  4 +-
 .../cassandra/db/compaction/CompactionTask.java | 81 ++--
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 +-
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 +-
 .../cassandra/db/lifecycle/TrackerTest.java |  6 +-
 .../org/apache/cassandra/tools/ToolsTester.java |  3 -
 10 files changed, 85 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9ead482/CHANGES.txt
--

[01/19] cassandra git commit: Temporarily fix bug that creates commit log when running offline tools

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 fb2940050 -> 66f1aaf88
  refs/heads/cassandra-3.0 e9b7a0f25 -> 0fe82be83
  refs/heads/cassandra-3.11 1cafc3ccd -> 66e214592
  refs/heads/cassandra-3.X ddefb1d89 -> c9ead4826
  refs/heads/trunk cd5aacc83 -> 402f8e006


Temporarily fix bug that creates commit log when running offline tools

patch by yukim; reviewed by thobbs for CASSANDRA-8616


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66f1aaf8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66f1aaf8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66f1aaf8

Branch: refs/heads/cassandra-2.2
Commit: 66f1aaf88d3cde5c52b13d71d3326da5eda16fb1
Parents: fb29400
Author: Yuki Morishita 
Authored: Thu Feb 11 19:06:27 2016 -0600
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:51:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/compaction/CompactionTask.java | 22 +++-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 36 
 .../io/sstable/format/SSTableReader.java|  8 +++--
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../io/sstable/CQLSSTableWriterLongTest.java|  2 ++
 .../unit/org/apache/cassandra/SchemaLoader.java |  1 +
 .../config/DatabaseDescriptorTest.java  |  6 
 .../apache/cassandra/db/CounterCellTest.java|  3 ++
 .../org/apache/cassandra/db/NativeCellTest.java |  3 ++
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../service/StorageServiceServerTest.java   |  1 +
 25 files changed, 118 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f02350d..8cff097 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
  * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
  * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
  * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3fb79b..35debd0 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -102,6 +102,18 @@ public class DatabaseDescriptor
 private static Comparator localComparator;
 private static boolean hasLoggedConfig;
 
+private static boolean daemonInitialized;
+
+public static boolean isDaemonInitialized()
+{
+return daemonInitialized;
+}
+
+public static void setDaemonInitialized()
+{
+daemonInitialized = true;
+}
+
 public static void forceStaticInitialization() {}
 static
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index c6b69dc..4bc46d0 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -388,9 +388,13 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 logger.info("Initializing {}.{}", keyspace.getName(), name);
 
-// scan for sstables corresponding to this cf and load them
-data = new Tracker(this, 

[14/19] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66e21459
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66e21459
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66e21459

Branch: refs/heads/trunk
Commit: 66e214592875e296bb540a966f1648f1106b2464
Parents: 1cafc3c 0fe82be
Author: Yuki Morishita 
Authored: Tue Dec 13 15:59:14 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:59:14 2016 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 ++-
 .../cassandra/db/PartitionRangeReadCommand.java |  4 +-
 .../cassandra/db/compaction/CompactionTask.java | 81 ++--
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 +-
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 +-
 .../cassandra/db/lifecycle/TrackerTest.java |  6 +-
 .../org/apache/cassandra/tools/ToolsTester.java |  3 -
 10 files changed, 85 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/CHANGES.txt
--
diff --cc CHANGES.txt
index 28ebf36,145afb9..f95dd81
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -168,12 -59,6 +168,13 @@@ Merged from 3.0
   * Correct log message for statistics of offheap memtable flush 
(CASSANDRA-12776)
   * Explicitly set locale for string validation 
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
  Merged from 2.2:
++ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
   * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
   * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
   * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 881fb00,39ed804..a5f76bd4
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -405,9 -388,13 +405,13 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+ Memtable initialMemtable = null;
+ if (DatabaseDescriptor.isDaemonInitialized())
 -initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
++initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getCurrentPosition()), this);
+ data = new Tracker(initialMemtable, loadSSTables);
  
+ // scan for sstables corresponding to this cf and load them
  if (data.loadsstables)
  {
  Directories.SSTableLister sstableFiles = 
directories.sstableLister(Directories.OnTxnErr.IGNORE).skipTemporary(true);
@@@ -2118,7 -1957,7 +2122,7 @@@
  {
  public Void call()
  {
- cfs.data.reset();
 -cfs.data.reset(new Memtable(new 
AtomicReference<>(ReplayPosition.NONE), cfs));
++cfs.data.reset(new Memtable(new 
AtomicReference<>(CommitLogPosition.NONE), cfs));
  return null;
  }
  }, true, false);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
--
diff --cc src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
index 50b568e,17adef0..045fc26
--- a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
+++ b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
@@@ -221,7 -199,8 +221,9 @@@ 

[15/19] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66e21459
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66e21459
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66e21459

Branch: refs/heads/cassandra-3.X
Commit: 66e214592875e296bb540a966f1648f1106b2464
Parents: 1cafc3c 0fe82be
Author: Yuki Morishita 
Authored: Tue Dec 13 15:59:14 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:59:14 2016 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 ++-
 .../cassandra/db/PartitionRangeReadCommand.java |  4 +-
 .../cassandra/db/compaction/CompactionTask.java | 81 ++--
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 +-
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 +-
 .../cassandra/db/lifecycle/TrackerTest.java |  6 +-
 .../org/apache/cassandra/tools/ToolsTester.java |  3 -
 10 files changed, 85 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/CHANGES.txt
--
diff --cc CHANGES.txt
index 28ebf36,145afb9..f95dd81
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -168,12 -59,6 +168,13 @@@ Merged from 3.0
   * Correct log message for statistics of offheap memtable flush 
(CASSANDRA-12776)
   * Explicitly set locale for string validation 
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
  Merged from 2.2:
++ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
   * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
   * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
   * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 881fb00,39ed804..a5f76bd4
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -405,9 -388,13 +405,13 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+ Memtable initialMemtable = null;
+ if (DatabaseDescriptor.isDaemonInitialized())
 -initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
++initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getCurrentPosition()), this);
+ data = new Tracker(initialMemtable, loadSSTables);
  
+ // scan for sstables corresponding to this cf and load them
  if (data.loadsstables)
  {
  Directories.SSTableLister sstableFiles = 
directories.sstableLister(Directories.OnTxnErr.IGNORE).skipTemporary(true);
@@@ -2118,7 -1957,7 +2122,7 @@@
  {
  public Void call()
  {
- cfs.data.reset();
 -cfs.data.reset(new Memtable(new 
AtomicReference<>(ReplayPosition.NONE), cfs));
++cfs.data.reset(new Memtable(new 
AtomicReference<>(CommitLogPosition.NONE), cfs));
  return null;
  }
  }, true, false);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
--
diff --cc src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
index 50b568e,17adef0..045fc26
--- a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
+++ b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
@@@ -221,7 -199,8 +221,9 

[16/19] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66e21459
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66e21459
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66e21459

Branch: refs/heads/cassandra-3.11
Commit: 66e214592875e296bb540a966f1648f1106b2464
Parents: 1cafc3c 0fe82be
Author: Yuki Morishita 
Authored: Tue Dec 13 15:59:14 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:59:14 2016 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 ++-
 .../cassandra/db/PartitionRangeReadCommand.java |  4 +-
 .../cassandra/db/compaction/CompactionTask.java | 81 ++--
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 +-
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 +-
 .../cassandra/db/lifecycle/TrackerTest.java |  6 +-
 .../org/apache/cassandra/tools/ToolsTester.java |  3 -
 10 files changed, 85 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/CHANGES.txt
--
diff --cc CHANGES.txt
index 28ebf36,145afb9..f95dd81
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -168,12 -59,6 +168,13 @@@ Merged from 3.0
   * Correct log message for statistics of offheap memtable flush 
(CASSANDRA-12776)
   * Explicitly set locale for string validation 
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
  Merged from 2.2:
++ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
   * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
   * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
   * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 881fb00,39ed804..a5f76bd4
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -405,9 -388,13 +405,13 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+ Memtable initialMemtable = null;
+ if (DatabaseDescriptor.isDaemonInitialized())
 -initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
++initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getCurrentPosition()), this);
+ data = new Tracker(initialMemtable, loadSSTables);
  
+ // scan for sstables corresponding to this cf and load them
  if (data.loadsstables)
  {
  Directories.SSTableLister sstableFiles = 
directories.sstableLister(Directories.OnTxnErr.IGNORE).skipTemporary(true);
@@@ -2118,7 -1957,7 +2122,7 @@@
  {
  public Void call()
  {
- cfs.data.reset();
 -cfs.data.reset(new Memtable(new 
AtomicReference<>(ReplayPosition.NONE), cfs));
++cfs.data.reset(new Memtable(new 
AtomicReference<>(CommitLogPosition.NONE), cfs));
  return null;
  }
  }, true, false);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66e21459/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
--
diff --cc src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
index 50b568e,17adef0..045fc26
--- a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
+++ b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java
@@@ -221,7 -199,8 +221,9 

[04/19] cassandra git commit: Temporarily fix bug that creates commit log when running offline tools

[yukim]

Temporarily fix bug that creates commit log when running offline tools

patch by yukim; reviewed by thobbs for CASSANDRA-8616


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66f1aaf8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66f1aaf8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66f1aaf8

Branch: refs/heads/cassandra-3.11
Commit: 66f1aaf88d3cde5c52b13d71d3326da5eda16fb1
Parents: fb29400
Author: Yuki Morishita 
Authored: Thu Feb 11 19:06:27 2016 -0600
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:51:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/compaction/CompactionTask.java | 22 +++-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 36 
 .../io/sstable/format/SSTableReader.java|  8 +++--
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../io/sstable/CQLSSTableWriterLongTest.java|  2 ++
 .../unit/org/apache/cassandra/SchemaLoader.java |  1 +
 .../config/DatabaseDescriptorTest.java  |  6 
 .../apache/cassandra/db/CounterCellTest.java|  3 ++
 .../org/apache/cassandra/db/NativeCellTest.java |  3 ++
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../service/StorageServiceServerTest.java   |  1 +
 25 files changed, 118 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f02350d..8cff097 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
  * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
  * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
  * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3fb79b..35debd0 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -102,6 +102,18 @@ public class DatabaseDescriptor
 private static Comparator localComparator;
 private static boolean hasLoggedConfig;
 
+private static boolean daemonInitialized;
+
+public static boolean isDaemonInitialized()
+{
+return daemonInitialized;
+}
+
+public static void setDaemonInitialized()
+{
+daemonInitialized = true;
+}
+
 public static void forceStaticInitialization() {}
 static
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f1aaf8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index c6b69dc..4bc46d0 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -388,9 +388,13 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 logger.info("Initializing {}.{}", keyspace.getName(), name);
 
-// scan for sstables corresponding to this cf and load them
-data = new Tracker(this, loadSSTables);
+// Create Memtable only on online
+Memtable initialMemtable = null;
+if (DatabaseDescriptor.isDaemonInitialized())
+initialMemtable = new Memtable(new 
AtomicReference<>(CommitLog.instance.getContext()), this);
+data = new 

[10/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --cc src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index 003b624,000..a6ed3ba
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@@ -1,1112 -1,0 +1,1114 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.index;
 +
 +import java.lang.reflect.Constructor;
 +import java.util.*;
 +import java.util.concurrent.*;
 +import java.util.function.Function;
 +import java.util.stream.Collectors;
 +import java.util.stream.Stream;
 +
 +import com.google.common.base.Joiner;
 +import com.google.common.base.Strings;
 +import com.google.common.collect.ImmutableSet;
 +import com.google.common.collect.Iterables;
 +import com.google.common.collect.Maps;
 +import com.google.common.collect.Sets;
 +import com.google.common.primitives.Longs;
 +import com.google.common.util.concurrent.Futures;
 +import com.google.common.util.concurrent.MoreExecutors;
 +import org.apache.commons.lang3.StringUtils;
 +import org.slf4j.Logger;
 +import org.slf4j.LoggerFactory;
 +
 +import org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor;
 +import org.apache.cassandra.concurrent.NamedThreadFactory;
 +import org.apache.cassandra.concurrent.StageManager;
 +import org.apache.cassandra.config.ColumnDefinition;
++import org.apache.cassandra.config.DatabaseDescriptor;
 +import org.apache.cassandra.cql3.statements.IndexTarget;
 +import org.apache.cassandra.db.*;
 +import org.apache.cassandra.db.compaction.CompactionManager;
 +import org.apache.cassandra.db.filter.RowFilter;
 +import org.apache.cassandra.db.lifecycle.SSTableSet;
 +import org.apache.cassandra.db.lifecycle.View;
 +import org.apache.cassandra.db.partitions.PartitionIterators;
 +import org.apache.cassandra.db.partitions.PartitionUpdate;
 +import org.apache.cassandra.db.rows.*;
 +import org.apache.cassandra.exceptions.InvalidRequestException;
 +import org.apache.cassandra.index.internal.CassandraIndex;
 +import org.apache.cassandra.index.transactions.*;
 +import org.apache.cassandra.io.sstable.ReducingKeyIterator;
 +import org.apache.cassandra.io.sstable.format.SSTableReader;
 +import org.apache.cassandra.schema.IndexMetadata;
 +import org.apache.cassandra.schema.Indexes;
 +import org.apache.cassandra.service.pager.SinglePartitionPager;
 +import org.apache.cassandra.tracing.Tracing;
 +import org.apache.cassandra.transport.Server;
 +import org.apache.cassandra.utils.FBUtilities;
 +import org.apache.cassandra.utils.concurrent.OpOrder;
 +import org.apache.cassandra.utils.concurrent.Refs;
 +
 +/**
 + * Handles the core maintenance functionality associated with indexes: 
adding/removing them to or from
 + * a table, (re)building during bootstrap or other streaming operations, 
flushing, reloading metadata
 + * and so on.
 + *
 + * The Index interface defines a number of methods which return Callable. 
These are primarily the
 + * management tasks for an index implementation. Most of them are currently 
executed in a blocking
 + * fashion via submission to SIM's blockingExecutor. This provides the 
desired behaviour in pretty
 + * much all cases, as tasks like flushing an index needs to be executed 
synchronously to avoid potentially
 + * deadlocking on the FlushWriter or PostFlusher. Several of these 
Callable returning methods on Index could
 + * then be defined with as void and called directly from SIM (rather than 
being run via the executor service).
 + * Separating the task defintion from execution gives us greater flexibility 
though, so that in future, for example,
 + * if the flush process allows it we leave open the possibility of executing 
more of these tasks asynchronously.
 + *
 + * The primary exception to the above is the Callable returned from 
Index#addIndexedColumn. This may
 + * involve a significant effort, building a new index over any existing data. 
We perform this task asynchronously;
 + * as it is called as part of a schema 

[09/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0fe82be8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0fe82be8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0fe82be8

Branch: refs/heads/cassandra-3.0
Commit: 0fe82be83cceceb12172d63913388678253413bc
Parents: e9b7a0f 66f1aaf
Author: Yuki Morishita 
Authored: Tue Dec 13 15:55:34 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:55:34 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/PartitionRangeReadCommand.java |  3 +-
 .../cassandra/db/compaction/CompactionTask.java | 18 +++
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 ++-
 .../io/sstable/format/SSTableReader.java|  2 +-
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../config/DatabaseDescriptorTest.java  |  6 
 .../org/apache/cassandra/cql3/CQLTester.java|  1 +
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../metrics/HintedHandOffMetricsTest.java   |  7 
 .../service/StorageServiceServerTest.java   |  1 +
 .../concurrent/AbstractTransactionalTest.java   |  7 
 27 files changed, 124 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/CHANGES.txt
--
diff --cc CHANGES.txt
index 5621c93,8cff097..145afb9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.9
 +3.0.11
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * CQL often queries static columns unnecessarily (CASSANDRA-12768)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
   * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
   * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
   * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 71e1653,4bc46d0..39ed804
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -388,13 -388,17 +388,17 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+   

[07/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0fe82be8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0fe82be8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0fe82be8

Branch: refs/heads/trunk
Commit: 0fe82be83cceceb12172d63913388678253413bc
Parents: e9b7a0f 66f1aaf
Author: Yuki Morishita 
Authored: Tue Dec 13 15:55:34 2016 -0800
Committer: Yuki Morishita 
Committed: Tue Dec 13 15:55:34 2016 -0800

--
 CHANGES.txt |  1 +
 .../cassandra/config/DatabaseDescriptor.java| 12 +++
 .../apache/cassandra/db/ColumnFamilyStore.java  | 10 --
 .../cassandra/db/PartitionRangeReadCommand.java |  3 +-
 .../cassandra/db/compaction/CompactionTask.java | 18 +++
 .../cassandra/db/lifecycle/LogTransaction.java  |  3 +-
 .../apache/cassandra/db/lifecycle/Tracker.java  | 34 
 .../cassandra/index/SecondaryIndexManager.java  |  4 ++-
 .../io/sstable/format/SSTableReader.java|  2 +-
 .../cassandra/service/CassandraDaemon.java  |  1 +
 .../service/EmbeddedCassandraService.java   |  2 ++
 .../config/DatabaseDescriptorTest.java  |  6 
 .../org/apache/cassandra/cql3/CQLTester.java|  1 +
 .../apache/cassandra/db/SystemKeyspaceTest.java |  2 ++
 .../db/context/CounterContextTest.java  |  8 +
 .../db/lifecycle/LifecycleTransactionTest.java  |  5 ++-
 .../cassandra/db/lifecycle/TrackerTest.java |  7 ++--
 .../cassandra/dht/StreamStateStoreTest.java |  7 
 .../cassandra/gms/FailureDetectorTest.java  |  2 ++
 .../org/apache/cassandra/gms/GossiperTest.java  |  5 +++
 .../io/sstable/CQLSSTableWriterTest.java|  2 ++
 .../cassandra/locator/CloudstackSnitchTest.java |  2 ++
 .../apache/cassandra/locator/EC2SnitchTest.java |  2 ++
 .../locator/GoogleCloudSnitchTest.java  |  2 ++
 .../metrics/HintedHandOffMetricsTest.java   |  7 
 .../service/StorageServiceServerTest.java   |  1 +
 .../concurrent/AbstractTransactionalTest.java   |  7 
 27 files changed, 124 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/CHANGES.txt
--
diff --cc CHANGES.txt
index 5621c93,8cff097..145afb9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.9
 +3.0.11
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * CQL often queries static columns unnecessarily (CASSANDRA-12768)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
   * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
   * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
   * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0fe82be8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 71e1653,4bc46d0..39ed804
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -388,13 -388,17 +388,17 @@@ public class ColumnFamilyStore implemen
  
  logger.info("Initializing {}.{}", keyspace.getName(), name);
  
- // scan for sstables corresponding to this cf and load them
- data = new Tracker(this, loadSSTables);
+ // Create Memtable only on online
+ 

[jira] [Comment Edited] (CASSANDRA-12793) invalid jvm type and architecture [cassandra-env.sh]

[Chris Kistner (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746339#comment-15746339
 ] 

Chris Kistner edited comment on CASSANDRA-12793 at 12/14/16 12:20 AM:
--

[~ApAdAnA], we're running the same Java version on CentOS 7, but with the 
[Cassandra 3.7 from the binary tar 
file|http://archive.apache.org/dist/cassandra/3.7/], and in that version the 
cassandra-env.sh seems to be correct:

{code}
java_ver_output=`"${JAVA:-java}" -version 2>&1`
jvmver=`echo "$java_ver_output" | grep '[openjdk|java] version' | awk -F'"' 
'NR==1 {print $2}' | cut -d\- -f1`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
{code}

So I suppose you installed Cassandra 3.7 via apt?


was (Author: padakwaak):
[~ApAdAnA], we're running the same Java version on CentOS 7, but with the 
[Cassandra 3.7 from the binary tar 
file|http://archive.apache.org/dist/cassandra/3.7/], and in that version the 
cassandra-env.sh seems to be correct:

{code}
java_ver_output=`"${JAVA:-java}" -version 2>&1`
jvmver=`echo "$java_ver_output" | grep '[openjdk|java] version' | awk -F'"' 
'NR==1 {print $2}'`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
{code}

So I suppose you installed Cassandra 3.7 via apt?

> invalid jvm type and architecture [cassandra-env.sh]
> 
>
> Key: CASSANDRA-12793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12793
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
> Environment: ubuntu 16.04, openjdk 1.8.0_91
>Reporter: Ali Ebrahiminejad
>Priority: Minor
> Fix For: 3.7
>
>
> In cassandra-env.sh the part that determines the type of JVM we'll be running 
> on doesn't provide the right answer for openjdk 1.8.0_91.
> value of java_ver_output is "openjdk version "1.8.0_91" OpenJDK Runtime 
> Environment (build 1.8.0_91-8u91-b14-3ubuntu1~16.04.1-b14) OpenJDK 64-Bit 
> Server VM (build 25.91-b14, mixed mode)", yet the command looks for "java 
> version" (jvm=`echo "$java_ver_output" | grep -A 1 'java version' ...) which 
> does not exist.
> I guess it should be replaced with jvm=`echo "$java_ver_output" | grep -A 1 
> '[openjdk|java] version' | awk 'NR==2 {print $1}'`



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746508#comment-15746508
 ] 

Corentin Chary commented on CASSANDRA-13040:


Additional fix related to this in CASSANDRA-13038, looks like this patch still 
update the histogram for cells with a ttl.

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 3.0.11, 3.10, 3.12, 4.0
>
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746508#comment-15746508
 ] 

Corentin Chary edited comment on CASSANDRA-13040 at 12/13/16 10:43 PM:
---

Additional fix related to this in CASSANDRA-13038, looks like this patch still 
update the tombstone histogram for cells with a ttl and this has a significant 
performance penalty.


was (Author: iksaif):
Additional fix related to this in CASSANDRA-13038, looks like this patch still 
update the histogram for cells with a ttl.

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 3.0.11, 3.10, 3.12, 4.0
>
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[Corentin Chary (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corentin Chary updated CASSANDRA-13038:
---
Status: Patch Available  (was: Open)

What about this patch ? Basically it also doesn't update the tombstone 
histogram for expiring cells. Another approach could simply be to update it if 
isTombstone() is true.

Still not perfect because it doesn't handle update(DeletionTime dt).

> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, 
> profiler-snapshot.nps, tombstone-histograms-expiring.patch
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[Corentin Chary (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corentin Chary updated CASSANDRA-13038:
---
Attachment: tombstone-histograms-expiring.patch

> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, 
> profiler-snapshot.nps, tombstone-histograms-expiring.patch
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13036) Row and Cell Level Authorisation

[Alexander McRae (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746488#comment-15746488
 ] 

Alexander McRae commented on CASSANDRA-13036:
-

Hi Stefan,

Thanks for the quick reply.

I did have a look at this JIRA previously and it would potentially solve the 
cell level where the user is restricted from seeing a column regardless of the 
context of the entity (record) being queried.

However I don't believe it doesn't solve the row level permission issue which 
is dependent of the context of the entity (record) being queried. For example 
the user may have permission to see one record in the table but not another. Or 
the permissions may vary between records, eg. on one it may have only SELECT 
but on another it may have MODIFY depending on the authority it has be given on 
the entity that the record relates to.

Let me know if that needs further clarification.

Regards,

Alex

> Row and Cell Level Authorisation
> 
>
> Key: CASSANDRA-13036
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13036
> Project: Cassandra
>  Issue Type: Wish
>  Components: Core
>Reporter: Alexander McRae
>
> Hi,
> I am wondering if there are plans to implement Row Level Authorisation. 
> The industry I work in his highly regulated and the security policies within 
> our company/industry require us to have some strict data access control 
> requirements which means that we need to do permission checking on a user at 
> a row level or potentially down to a cell (attribute) level.
> I have had a quick look through the documentation, code and jira tickets and 
> it doesn't appear that level of authorisation is catered for or is currently 
> being considered.
> As such I would like request to see if this feature could possibly listed as 
> future enhancement.
> Regards,
> Alex 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12793) invalid jvm type and architecture [cassandra-env.sh]

[Chris Kistner (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746339#comment-15746339
 ] 

Chris Kistner commented on CASSANDRA-12793:
---

[~ApAdAnA], we're running the same Java version on CentOS 7, but with the 
[Cassandra 3.7 from the binary tar 
file|http://archive.apache.org/dist/cassandra/3.7/], and in that version the 
cassandra-env.sh seems to be correct:

{code}
java_ver_output=`"${JAVA:-java}" -version 2>&1`
jvmver=`echo "$java_ver_output" | grep '[openjdk|java] version' | awk -F'"' 
'NR==1 {print $2}'`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
{code}

So I suppose you installed Cassandra 3.7 via apt?

> invalid jvm type and architecture [cassandra-env.sh]
> 
>
> Key: CASSANDRA-12793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12793
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
> Environment: ubuntu 16.04, openjdk 1.8.0_91
>Reporter: Ali Ebrahiminejad
>Priority: Minor
> Fix For: 3.7
>
>
> In cassandra-env.sh the part that determines the type of JVM we'll be running 
> on doesn't provide the right answer for openjdk 1.8.0_91.
> value of java_ver_output is "openjdk version "1.8.0_91" OpenJDK Runtime 
> Environment (build 1.8.0_91-8u91-b14-3ubuntu1~16.04.1-b14) OpenJDK 64-Bit 
> Server VM (build 25.91-b14, mixed mode)", yet the command looks for "java 
> version" (jvm=`echo "$java_ver_output" | grep -A 1 'java version' ...) which 
> does not exist.
> I guess it should be replaced with jvm=`echo "$java_ver_output" | grep -A 1 
> '[openjdk|java] version' | awk 'NR==2 {print $1}'`



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12186) anticompaction log message doesn't include the parent repair session id

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12186:

Status: Ready to Commit  (was: Patch Available)

> anticompaction log message doesn't include the parent repair session id
> ---
>
> Key: CASSANDRA-12186
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12186
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Wei Deng
>Assignee: Tommy Stendahl
>Priority: Minor
>  Labels: lhf
> Fix For: 3.x
>
> Attachments: 12186.txt
>
>
> It appears that even though incremental repair is now enabled by default post 
> C*-3.0 (which means at the end of each repair session, there is an 
> anti-compaction step that needs to be executed), we don't include the parent 
> repair session UUID in the log message of the anti-compaction log entries. 
> This makes observing all activities related to an incremental repair session 
> to be more difficult. See the following:
> {noformat}
> DEBUG [AntiEntropyStage:1] 2016-07-13 01:57:30,956  
> RepairMessageVerbHandler.java:149 - Got anticompaction request 
> AnticompactionRequest{parentRepairSession=27103de0-489d-11e6-a6d6-cd06faa0aaa2}
>  org.apache.cassandra.repair.messages.AnticompactionRequest@34449ff4
> <...>
> 
> <...>
> INFO  [CompactionExecutor:5] 2016-07-13 02:07:47,512  
> CompactionManager.java:511 - Starting anticompaction for trivial_ks.weitest 
> on 
> 1/[BigTableReader(path='/var/lib/cassandra/data/trivial_ks/weitest-538b07d1489b11e6a9ef61c6ff848952/mb-1-big-Data.db')]
>  sstables
> INFO  [CompactionExecutor:5] 2016-07-13 02:07:47,513  
> CompactionManager.java:540 - SSTable 
> BigTableReader(path='/var/lib/cassandra/data/trivial_ks/weitest-538b07d1489b11e6a9ef61c6ff848952/mb-1-big-Data.db')
>  fully contained in range (-9223372036854775808,-9223372036854775808], 
> mutating repairedAt instead of anticompacting
> INFO  [CompactionExecutor:5] 2016-07-13 02:07:47,570  
> CompactionManager.java:578 - Completed anticompaction successfully
> {noformat}
> The initial submission of the anti-compaction task to the CompactionManager 
> still has reference to the parent repair session UUID, but subsequent 
> anti-compaction log entries are missing this parent repair session UUID.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12510) Disallow decommission when number of replicas will drop below configured RF

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12510:

Status: Patch Available  (was: Open)

> Disallow decommission when number of replicas will drop below configured RF
> ---
>
> Key: CASSANDRA-12510
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12510
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
> Environment: C* version 3.3
>Reporter: Atin Sood
>Assignee: Kurt Greaves
>Priority: Minor
>  Labels: lhf
> Attachments: 12510-3.x.patch
>
>
> Steps to replicate :
> - Create a 3 node cluster in DC1 and create a keyspace test_keyspace with 
> table test_table with replication strategy NetworkTopologyStrategy , DC1=3 . 
> Populate some data into this table.
> - Add 5 more nodes to this cluster, but in DC2. Also do not alter the 
> keyspace to add the new DC2 to replication (this is intentional and the 
> reason why the bug shows up). So the desc keyspace should still list 
> NetworkTopologyStrategy with DC1=3 as RF
> - As expected, this will now be a 8 node cluster with 3 nodes in DC1 and 5 in 
> DC2
> - Now start decommissioning the nodes in DC1. Note that the decommission runs 
> fine on all the 3 nodes, but since the new nodes are in DC2 and the RF for 
> keyspace is restricted to DC1, the new 5 nodes won't get any data.
> - You will now end with the 5 node cluster which has no data from the 
> decommissioned 3 nodes and hence ending up in data loss
> I do understand that this problem could have been avoided if we perform an 
> alter stmt and add DC2 replication before adding the 5 nodes. But the fact 
> that decommission ran fine on the 3 nodes on DC1 without complaining that 
> there were no nodes to stream its data seems a little discomforting. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12984) MVs are built unnecessarily again after bootstrap

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-12984:
---
   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 3.x)
   3.10
   3.0.11
   Status: Resolved  (was: Patch Available)

Thanks for the patch, [~brstgt]! Tests looked reasonable, so I committed the 
patch as 
[e9b7a0f|https://git1-us-west.apache.org/repos/asf/cassandra/?p=cassandra.git;a=commit;h=e9b7a0f2546579244ffc167c56122b0a47d4b4b0].

> MVs are built unnecessarily again after bootstrap
> -
>
> Key: CASSANDRA-12984
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12984
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
> Fix For: 3.0.11, 3.10
>
>
> After bootstrap MVs are enqueued to be built but they have been already 
> created by the bootstrap.
> Simply adding them to system.built_views after a successful bootstrap should 
> fix that issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[03/10] cassandra git commit: Mark MVs as built after successful bootstrap

[carl]

Mark MVs as built after successful bootstrap

Patch by Benjamin Roth; reviewed by Carl Yeksigian for CASSANDRA-12984


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9b7a0f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9b7a0f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9b7a0f2

Branch: refs/heads/cassandra-3.X
Commit: e9b7a0f2546579244ffc167c56122b0a47d4b4b0
Parents: b8ed255
Author: brstgt 
Authored: Fri Dec 2 09:41:20 2016 +0100
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:27:08 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 38660a0..5621c93 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
  * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 71cbc35..35b2423 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -57,6 +57,7 @@ import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.ViewDefinition;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.compaction.CompactionManager;
@@ -1211,7 +1212,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 logger.info("Bootstrap completed! for the tokens {}", tokens);
 }
 
@@ -1233,6 +1234,25 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 
+/**
+ * All MVs have been created during bootstrap, so mark them as built
+ */
+private void markViewsAsBuilt() {
+for (String keyspace : Schema.instance.getUserKeyspaces())
+{
+for (ViewDefinition view: 
Schema.instance.getKSMetaData(keyspace).views)
+SystemKeyspace.finishViewBuildStatus(view.ksName, 
view.viewName);
+}
+}
+
+/**
+ * Called when bootstrap did finish successfully
+ */
+private void bootstrapFinished() {
+markViewsAsBuilt();
+isBootstrapMode = false;
+}
+
 public boolean resumeBootstrap()
 {
 if (isBootstrapMode && SystemKeyspace.bootstrapInProgress())
@@ -1250,7 +1270,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 // start participating in the ring.
 // pretend we are in survey mode so we can use joinRing() 
here
 isSurveyMode = true;

[06/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[carl]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1cafc3cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1cafc3cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1cafc3cc

Branch: refs/heads/cassandra-3.11
Commit: 1cafc3ccd23049f225d2ce16da99338445184e77
Parents: 30e154d e9b7a0f
Author: Carl Yeksigian 
Authored: Tue Dec 13 12:29:01 2016 -0500
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:29:01 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cafc3cc/CHANGES.txt
--
diff --cc CHANGES.txt
index 953b720,5621c93..28ebf36
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add 

[02/10] cassandra git commit: Mark MVs as built after successful bootstrap

[carl]

Mark MVs as built after successful bootstrap

Patch by Benjamin Roth; reviewed by Carl Yeksigian for CASSANDRA-12984


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9b7a0f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9b7a0f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9b7a0f2

Branch: refs/heads/cassandra-3.11
Commit: e9b7a0f2546579244ffc167c56122b0a47d4b4b0
Parents: b8ed255
Author: brstgt 
Authored: Fri Dec 2 09:41:20 2016 +0100
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:27:08 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 38660a0..5621c93 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
  * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 71cbc35..35b2423 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -57,6 +57,7 @@ import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.ViewDefinition;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.compaction.CompactionManager;
@@ -1211,7 +1212,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 logger.info("Bootstrap completed! for the tokens {}", tokens);
 }
 
@@ -1233,6 +1234,25 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 
+/**
+ * All MVs have been created during bootstrap, so mark them as built
+ */
+private void markViewsAsBuilt() {
+for (String keyspace : Schema.instance.getUserKeyspaces())
+{
+for (ViewDefinition view: 
Schema.instance.getKSMetaData(keyspace).views)
+SystemKeyspace.finishViewBuildStatus(view.ksName, 
view.viewName);
+}
+}
+
+/**
+ * Called when bootstrap did finish successfully
+ */
+private void bootstrapFinished() {
+markViewsAsBuilt();
+isBootstrapMode = false;
+}
+
 public boolean resumeBootstrap()
 {
 if (isBootstrapMode && SystemKeyspace.bootstrapInProgress())
@@ -1250,7 +1270,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 // start participating in the ring.
 // pretend we are in survey mode so we can use joinRing() 
here
 isSurveyMode = true;

[10/10] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[carl]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cd5aacc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cd5aacc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cd5aacc8

Branch: refs/heads/trunk
Commit: cd5aacc83bc3ee2b412c191a8ce711b0401bf55a
Parents: d596a47 ddefb1d
Author: Carl Yeksigian 
Authored: Tue Dec 13 12:29:18 2016 -0500
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:29:18 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cd5aacc8/CHANGES.txt
--
diff --cc CHANGES.txt
index 1f18fd9,14ee562..83b9082
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -133,8 -123,9 +133,9 @@@
   * Remove pre-startup check for open JMX port (CASSANDRA-12074)
   * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738)
   * Restore resumable hints delivery (CASSANDRA-11960)
 - * Properly report LWT contention (CASSANDRA-12626)
 + * Properly record CAS contention (CASSANDRA-12626)
  Merged from 3.0:
+  * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
   * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
   * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
   * Thread local pools never cleaned up (CASSANDRA-13033)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cd5aacc8/src/java/org/apache/cassandra/service/StorageService.java
--

[07/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[carl]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1cafc3cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1cafc3cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1cafc3cc

Branch: refs/heads/trunk
Commit: 1cafc3ccd23049f225d2ce16da99338445184e77
Parents: 30e154d e9b7a0f
Author: Carl Yeksigian 
Authored: Tue Dec 13 12:29:01 2016 -0500
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:29:01 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cafc3cc/CHANGES.txt
--
diff --cc CHANGES.txt
index 953b720,5621c93..28ebf36
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH 

[08/10] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[carl]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ddefb1d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ddefb1d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ddefb1d8

Branch: refs/heads/cassandra-3.X
Commit: ddefb1d895f71060236f31cd1e91168d35e436d4
Parents: 707c364 1cafc3c
Author: Carl Yeksigian 
Authored: Tue Dec 13 12:29:11 2016 -0500
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:29:11 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ddefb1d8/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ddefb1d8/src/java/org/apache/cassandra/service/StorageService.java
--

[09/10] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[carl]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ddefb1d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ddefb1d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ddefb1d8

Branch: refs/heads/trunk
Commit: ddefb1d895f71060236f31cd1e91168d35e436d4
Parents: 707c364 1cafc3c
Author: Carl Yeksigian 
Authored: Tue Dec 13 12:29:11 2016 -0500
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:29:11 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ddefb1d8/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ddefb1d8/src/java/org/apache/cassandra/service/StorageService.java
--

[05/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[carl]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1cafc3cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1cafc3cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1cafc3cc

Branch: refs/heads/cassandra-3.X
Commit: 1cafc3ccd23049f225d2ce16da99338445184e77
Parents: 30e154d e9b7a0f
Author: Carl Yeksigian 
Authored: Tue Dec 13 12:29:01 2016 -0500
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:29:01 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cafc3cc/CHANGES.txt
--
diff --cc CHANGES.txt
index 953b720,5621c93..28ebf36
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH 

[04/10] cassandra git commit: Mark MVs as built after successful bootstrap

[carl]

Mark MVs as built after successful bootstrap

Patch by Benjamin Roth; reviewed by Carl Yeksigian for CASSANDRA-12984


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9b7a0f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9b7a0f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9b7a0f2

Branch: refs/heads/trunk
Commit: e9b7a0f2546579244ffc167c56122b0a47d4b4b0
Parents: b8ed255
Author: brstgt 
Authored: Fri Dec 2 09:41:20 2016 +0100
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:27:08 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 38660a0..5621c93 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
  * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 71cbc35..35b2423 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -57,6 +57,7 @@ import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.ViewDefinition;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.compaction.CompactionManager;
@@ -1211,7 +1212,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 logger.info("Bootstrap completed! for the tokens {}", tokens);
 }
 
@@ -1233,6 +1234,25 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 
+/**
+ * All MVs have been created during bootstrap, so mark them as built
+ */
+private void markViewsAsBuilt() {
+for (String keyspace : Schema.instance.getUserKeyspaces())
+{
+for (ViewDefinition view: 
Schema.instance.getKSMetaData(keyspace).views)
+SystemKeyspace.finishViewBuildStatus(view.ksName, 
view.viewName);
+}
+}
+
+/**
+ * Called when bootstrap did finish successfully
+ */
+private void bootstrapFinished() {
+markViewsAsBuilt();
+isBootstrapMode = false;
+}
+
 public boolean resumeBootstrap()
 {
 if (isBootstrapMode && SystemKeyspace.bootstrapInProgress())
@@ -1250,7 +1270,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 // start participating in the ring.
 // pretend we are in survey mode so we can use joinRing() 
here
 isSurveyMode = true;

[01/10] cassandra git commit: Mark MVs as built after successful bootstrap

[carl]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 b8ed255fd -> e9b7a0f25
  refs/heads/cassandra-3.11 30e154d3d -> 1cafc3ccd
  refs/heads/cassandra-3.X 707c36430 -> ddefb1d89
  refs/heads/trunk d596a47cf -> cd5aacc83


Mark MVs as built after successful bootstrap

Patch by Benjamin Roth; reviewed by Carl Yeksigian for CASSANDRA-12984


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9b7a0f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9b7a0f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9b7a0f2

Branch: refs/heads/cassandra-3.0
Commit: e9b7a0f2546579244ffc167c56122b0a47d4b4b0
Parents: b8ed255
Author: brstgt 
Authored: Fri Dec 2 09:41:20 2016 +0100
Committer: Carl Yeksigian 
Committed: Tue Dec 13 12:27:08 2016 -0500

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   | 24 ++--
 2 files changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 38660a0..5621c93 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
  * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9b7a0f2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 71cbc35..35b2423 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -57,6 +57,7 @@ import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.ViewDefinition;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.compaction.CompactionManager;
@@ -1211,7 +1212,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 logger.info("Bootstrap completed! for the tokens {}", tokens);
 }
 
@@ -1233,6 +1234,25 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 
+/**
+ * All MVs have been created during bootstrap, so mark them as built
+ */
+private void markViewsAsBuilt() {
+for (String keyspace : Schema.instance.getUserKeyspaces())
+{
+for (ViewDefinition view: 
Schema.instance.getKSMetaData(keyspace).views)
+SystemKeyspace.finishViewBuildStatus(view.ksName, 
view.viewName);
+}
+}
+
+/**
+ * Called when bootstrap did finish successfully
+ */
+private void bootstrapFinished() {
+markViewsAsBuilt();
+isBootstrapMode = false;
+}
+
 public boolean resumeBootstrap()
 {
 if (isBootstrapMode && SystemKeyspace.bootstrapInProgress())
@@ -1250,7 +1270,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onSuccess(StreamState streamState)
 {
-isBootstrapMode = false;
+bootstrapFinished();
 // start participating in the ring.
 // pretend we are in survey mode so we can use joinRing() 
here
 isSurveyMode = true;

[jira] [Updated] (CASSANDRA-11471) Add SASL mechanism negotiation to the native protocol

[Ben Bromhead (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ben Bromhead updated CASSANDRA-11471:
-
Attachment: CASSANDRA-11471

First cut of support for SASL negotiation in V4 for comment. 

The Authenticate message returns a comma seperated list of SASL mechanisms 
supported by the IAuthenticator class. Backwards compatible with V3 (returns 
classname instead).

Currently does not support detection of downgrade attacks as there is no 
integrity mechanisms. I think it's sufficient to rely on SSL for this.

Currently up to the IAuthenticator implementation to handle client selection of 
SASL mechanism (should be in the first response from client).

I have not updated CQL spec in this patch either. 

> Add SASL mechanism negotiation to the native protocol
> -
>
> Key: CASSANDRA-11471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11471
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Sam Tunnicliffe
>  Labels: client-impacting
> Attachments: CASSANDRA-11471
>
>
> Introducing an additional message exchange into the authentication sequence 
> would allow us to support multiple authentication schemes and [negotiation of 
> SASL mechanisms|https://tools.ietf.org/html/rfc4422#section-3.2]. 
> The current {{AUTHENTICATE}} message sent from Client to Server includes the 
> java classname of the configured {{IAuthenticator}}. This could be superceded 
> by a new message which lists the SASL mechanisms supported by the server. The 
> client would then respond with a new message which indicates it's choice of 
> mechanism.  This would allow the server to support multiple mechanisms, for 
> example enabling both {{PLAIN}} for username/password authentication and 
> {{EXTERNAL}} for a mechanism for extracting credentials from SSL 
> certificates\* (see the example in 
> [RFC-4422|https://tools.ietf.org/html/rfc4422#appendix-A]). Furthermore, the 
> server could tailor the list of supported mechanisms on a per-connection 
> basis, e.g. only offering certificate based auth to encrypted clients. 
> The client's response should include the selected mechanism and any initial 
> response data. This is mechanism-specific; the {{PLAIN}} mechanism consists 
> of a single round in which the client sends encoded credentials as the 
> initial response data and the server response indicates either success or 
> failure with no futher challenges required.
> From a protocol perspective, after the mechanism negotiation the exchange 
> would continue as in protocol v4, with one or more rounds of 
> {{AUTH_CHALLENGE}} and {{AUTH_RESPONSE}} messages, terminated by an 
> {{AUTH_SUCCESS}} sent from Server to Client upon successful authentication or 
> an {{ERROR}} on auth failure. 
> XMPP performs mechanism negotiation in this way, 
> [RFC-3920|http://tools.ietf.org/html/rfc3920#section-6] includes a good 
> overview.
> \* Note: this would require some a priori agreement between client and server 
> over the implementation of the {{EXTERNAL}} mechanism.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra-builds git commit: Add dtest variation jobs to DSL

[mshuler]

Repository: cassandra-builds
Updated Branches:
  refs/heads/master dac4ddfb9 -> bb0105561


Add dtest variation jobs to DSL


Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/bb010556
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/bb010556
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/bb010556

Branch: refs/heads/master
Commit: bb010556128d7e62bfc428143a03b77ce897addd
Parents: dac4ddf
Author: Michael Shuler 
Authored: Tue Dec 13 11:43:39 2016 -0600
Committer: Michael Shuler 
Committed: Tue Dec 13 11:43:39 2016 -0600

--
 jenkins-dsl/cassandra_job_dsl_seed.groovy | 29 +-
 1 file changed, 19 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/bb010556/jenkins-dsl/cassandra_job_dsl_seed.groovy
--
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy 
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 2c56f38..02bf485 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -15,6 +15,7 @@ def buildDescStr = 'REF = ${GIT_BRANCH}  COMMIT = 
${GIT_COMMIT}'
 def cassandraBranches = ['cassandra-2.2', 'cassandra-3.0', 'cassandra-3.11', 
'cassandra-3.X', 'trunk']
 // Ant test targets
 def testTargets = ['test', 'test-all', 'test-burn', 'test-cdc', 
'test-compression']
+def dtestTargets = ['dtest', 'dtest-novnode', 'dtest-offheap']  // dtest-large 
target exists, but no large servers to run on..
 
 
 //
@@ -205,7 +206,7 @@ cassandraBranches.each {
 
 // Skip test-cdc on cassandra-2.2 and cassandra-3.0 branches
 if ((targetName == 'test-cdc') && ((branchName == 'cassandra-2.2') || 
(branchName == 'cassandra-3.0'))) {
-println("Skipping ${targetName} on branch ${branchName}");
+println("Skipping ${targetName} on branch ${branchName}")
 } else {
  job("${jobNamePrefix}-${targetName}") {
 disabled(false)
@@ -223,15 +224,23 @@ cassandraBranches.each {
 /**
  * Main branch dtest variation jobs
  */
-// TODO: set up variations similar to unittest above, ie. novnodes - 
currently, this is a default dtest run for each branch
-job("${jobNamePrefix}-dtest") {
-disabled(false)
-using('Cassandra-template-dtest')
-configure { node ->
-node / scm / branches / 'hudson.plugins.git.BranchSpec' / 
name(branchName)
-}
-steps {
-shell("./cassandra-builds/build-scripts/cassandra-dtest.sh")
+dtestTargets.each {
+def targetName = it
+
+// Skip dtest-offheap on cassandra-3.0 branch
+if ((targetName == 'dtest-offheap') && (branchName == 
'cassandra-3.0')) {
+println("Skipping ${targetName} on branch ${branchName}")
+} else {
+job("${jobNamePrefix}-${targetName}") {
+disabled(false)
+using('Cassandra-template-dtest')
+configure { node ->
+node / scm / branches / 'hudson.plugins.git.BranchSpec' / 
name(branchName)
+}
+steps {
+shell("./cassandra-builds/build-scripts/cassandra-dtest.sh 
${targetName}")
+}
+}
 }
 }
 

cassandra-builds git commit: Add dtest variations to build script

[mshuler]

Repository: cassandra-builds
Updated Branches:
  refs/heads/master 344b6d227 -> dac4ddfb9


Add dtest variations to build script


Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/dac4ddfb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/dac4ddfb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/dac4ddfb

Branch: refs/heads/master
Commit: dac4ddfb9e065b014ab19d637b0ae0489e64df72
Parents: 344b6d2
Author: Michael Shuler 
Authored: Tue Dec 13 11:18:10 2016 -0600
Committer: Michael Shuler 
Committed: Tue Dec 13 11:18:10 2016 -0600

--
 build-scripts/cassandra-dtest.sh | 25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/dac4ddfb/build-scripts/cassandra-dtest.sh
--
diff --git a/build-scripts/cassandra-dtest.sh b/build-scripts/cassandra-dtest.sh
index 9a65e9b..d1bdd4b 100755
--- a/build-scripts/cassandra-dtest.sh
+++ b/build-scripts/cassandra-dtest.sh
@@ -6,6 +6,9 @@
 #
 
 
+# Pass in target to run, default to base dtest
+DTEST_TARGET="${1:-dtest}"
+
 export PYTHONIOENCODING="utf-8"
 export PYTHONUNBUFFERED=true
 export CASS_DRIVER_NO_EXTENSIONS=true
@@ -23,6 +26,11 @@ for x in $(seq 1 3); do
 break
 fi
 done
+# Exit, if we didn't build successfully
+if [ "${RETURN}" -ne "0" ]; then
+echo "Build failed with exit code: ${RETURN}"
+exit ${RETURN}
+fi
 
 # Set up venv with dtest dependencies
 set -e # enable immediate exit if venv setup fails
@@ -38,9 +46,22 @@ pip freeze
 
 
 cd cassandra-dtest/
-rm -r upgrade_tests/ # TEMP: remove upgrade_tests
+rm -r upgrade_tests/ # TEMP: remove upgrade_tests - we have no dual JDK 
installation
 set +e # disable immediate exit from this point
-./run_dtests.py --vnodes true --nose-options="--verbosity=3 --with-xunit 
--nocapture --attr=!resource-intensive" | tee -a ${WORKSPACE}/test_stdout.txt
+if [ "${DTEST_TARGET}" = "dtest" ]; then
+./run_dtests.py --vnodes true --nose-options="--verbosity=3 --with-xunit 
--nocapture --attr=!resource-intensive" | tee -a ${WORKSPACE}/test_stdout.txt
+elif [ "${DTEST_TARGET}" = "dtest-novnode" ]; then
+export DISABLE_VNODES="true"
+./run_dtests.py --vnodes true --nose-options="--verbosity=3 --with-xunit 
--nocapture --attr=!resource-intensive" | tee -a ${WORKSPACE}/test_stdout.txt
+elif [ "${DTEST_TARGET}" = "dtest-offheap" ]; then
+export OFFHEAP_MEMTABLES="true"
+./run_dtests.py --vnodes true --nose-options="--verbosity=3 --with-xunit 
--nocapture --attr=!resource-intensive" | tee -a ${WORKSPACE}/test_stdout.txt
+elif [ "${DTEST_TARGET}" = "dtest-large" ]; then
+./run_dtests.py --vnodes true --nose-options="--verbosity=3 --with-xunit 
--nocapture --attr=resource-intensive" | tee -a ${WORKSPACE}/test_stdout.txt
+else
+echo "Unknown dtest target: ${DTEST_TARGET}"
+exit 1
+fi
 
 
 #

[jira] [Commented] (CASSANDRA-8616) sstable tools may result in commit log segments be written

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745642#comment-15745642
 ] 

Tyler Hobbs commented on CASSANDRA-8616:


+1

> sstable tools may result in commit log segments be written
> --
>
> Key: CASSANDRA-8616
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8616
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Yuki Morishita
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 8161-2.0.txt
>
>
> There was a report of sstable2json causing commitlog segments to be written 
> out when run.  I haven't attempted to reproduce this yet, so that's all I 
> know for now.  Since sstable2json loads the conf and schema, I'm thinking 
> that it may inadvertently be triggering the commitlog code.
> sstablescrub, sstableverify, and other sstable tools have the same issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13034) Move to FastThreadLocalThread and FastThreadLocal

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-13034:
-
   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.12
   Status: Resolved  (was: Ready to Commit)

Thanks for the prompt review!

CI looks good.

Committed as 
[707c36430918b27f0c392dc0bf077af23be06c70|https://github.com/apache/cassandra/commit/707c36430918b27f0c392dc0bf077af23be06c70]
 to [cassandra-3.X|https://github.com/apache/cassandra/tree/cassandra-3.X] and 
merged to trunk.

> Move to FastThreadLocalThread and FastThreadLocal
> -
>
> Key: CASSANDRA-13034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13034
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.12
>
>
> (Supersedes/includes CASSANDRA-13033 for 3.X & trunk)
> We still use {{ThreadLocal}} in a couple of places, so I was curious how much 
> faster {{FastThreadLocal}} is compared to {{ThreadLocal}}. A micro bench 
> tells, that {{FastThreadLocal}} has a runtime of ~2.7ns and {{ThreadLocal}} 
> of ~4.7ns - about 2ns slower (EDIT: subtracted baseline).
> However, looking at the implementations it seems that {{ThreadLocal}} has 
> more dependent pointer gets than {{FastThreadLocal}}. This (CPU cache misses) 
> is not reflected in the artificial benchmark below.
> The patch migrates all {{Thread}} instances (except a few in tests) and all 
> {{ThreadLocal}} instances.
> {code:title=FastThreadLocalBench with 4 threads on 4 core CPU}
>  [java] FastThreadLocalBench.baseline   2  avgt5  
> 3.023 ± 0.081  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal2  avgt5  
> 5.610 ± 0.154  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal4  avgt5  
> 5.653 ± 0.042  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal8  avgt5  
> 5.763 ± 0.588  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal   12  avgt5  
> 5.673 ± 0.117  ns/op
>  [java] FastThreadLocalBench.threadLocal2  avgt5  
> 7.708 ± 0.723  ns/op
>  [java] FastThreadLocalBench.threadLocal4  avgt5  
> 7.604 ± 0.059  ns/op
>  [java] FastThreadLocalBench.threadLocal8  avgt5  
> 7.629 ± 0.080  ns/op
>  [java] FastThreadLocalBench.threadLocal   12  avgt5  
> 7.858 ± 0.483  ns/op
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Move to FastThreadLocalThread and FastThreadLocal

[snazy]

Move to FastThreadLocalThread and FastThreadLocal

patch by Robert Stupp; reviewed by Jason Brown for CASSANDRA-13034


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/707c3643
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/707c3643
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/707c3643

Branch: refs/heads/trunk
Commit: 707c36430918b27f0c392dc0bf077af23be06c70
Parents: 55adebe
Author: Robert Stupp 
Authored: Tue Dec 13 17:37:09 2016 +0100
Committer: Robert Stupp 
Committed: Tue Dec 13 17:37:47 2016 +0100

--
 CHANGES.txt |  1 +
 .../concurrent/NamedThreadFactory.java  | 42 -
 .../functions/ThreadAwareSecurityManager.java   |  3 +-
 .../AbstractCommitLogSegmentManager.java|  2 +-
 .../db/commitlog/AbstractCommitLogService.java  |  2 +-
 .../db/compaction/CompactionManager.java|  3 +-
 .../apache/cassandra/db/marshal/AsciiType.java  |  3 +-
 .../hints/EncryptedChecksummedDataInput.java|  3 +-
 .../cassandra/index/sasi/TermIterator.java  |  5 +-
 .../io/compress/DeflateCompressor.java  | 11 +--
 .../cassandra/net/OutboundTcpConnection.java|  5 +-
 .../apache/cassandra/repair/RepairRunnable.java |  6 +-
 .../scheduler/RoundRobinScheduler.java  |  2 +-
 .../cassandra/security/CipherFactory.java   |  3 +-
 .../cassandra/security/EncryptionUtils.java |  3 +-
 .../serializers/TimestampSerializer.java|  7 +-
 .../cassandra/service/StorageService.java   |  8 +-
 .../cassandra/thrift/ThriftSessionManager.java  |  4 +-
 .../cassandra/utils/CoalescingStrategies.java   | 25 +++---
 .../org/apache/cassandra/utils/FBUtilities.java | 15 ++--
 .../org/apache/cassandra/cql3/ViewLongTest.java |  3 +-
 .../test/microbench/FastThreadExecutor.java | 65 +-
 .../test/microbench/FastThreadLocalBench.java   | 92 
 .../cassandra/cache/CacheProviderTest.java  |  3 +-
 .../cassandra/concurrent/WaitQueueTest.java |  4 +-
 .../cassandra/db/RecoveryManagerTest.java   | 22 +++--
 .../apache/cassandra/hints/HintsBufferTest.java |  3 +-
 .../io/sstable/IndexSummaryManagerTest.java |  3 +-
 .../io/sstable/SSTableRewriterTest.java |  5 +-
 .../apache/cassandra/service/RemoveTest.java| 26 +++---
 .../apache/cassandra/utils/TopKSamplerTest.java |  5 +-
 .../apache/cassandra/stress/StressServer.java   |  6 +-
 .../operations/userdefined/TokenRangeQuery.java |  3 +-
 33 files changed, 238 insertions(+), 155 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/707c3643/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a225df8..2c95d50 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.12
+ * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
  * nodetool stopdaemon errors out (CASSANDRA-13030)
  * Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028)
  * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/707c3643/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java 
b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
index 22193c4..5d89f6c 100644
--- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
@@ -20,6 +20,8 @@ package org.apache.cassandra.concurrent;
 import java.util.concurrent.ThreadFactory;
 import java.util.concurrent.atomic.AtomicInteger;
 
+import com.google.common.annotations.VisibleForTesting;
+
 import io.netty.util.concurrent.FastThreadLocal;
 import io.netty.util.concurrent.FastThreadLocalThread;
 
@@ -58,9 +60,8 @@ public class NamedThreadFactory implements ThreadFactory
 public Thread newThread(Runnable runnable)
 {
 String name = id + ':' + n.getAndIncrement();
-Thread thread = new FastThreadLocalThread(threadGroup, 
threadLocalDeallocator(runnable), name);
+Thread thread = createThread(threadGroup, runnable, name, true);
 thread.setPriority(priority);
-thread.setDaemon(true);
 if (contextClassLoader != null)
 thread.setContextClassLoader(contextClassLoader);
 return thread;
@@ -75,11 +76,44 @@ public class NamedThreadFactory implements ThreadFactory
 {
 return () ->
 {
-try {
+try
+{
 r.run();
-} finally {
+}
+finally
+{

[1/3] cassandra git commit: Move to FastThreadLocalThread and FastThreadLocal

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.X 55adebe66 -> 707c36430
  refs/heads/trunk 4489ee8d8 -> d596a47cf


Move to FastThreadLocalThread and FastThreadLocal

patch by Robert Stupp; reviewed by Jason Brown for CASSANDRA-13034


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/707c3643
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/707c3643
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/707c3643

Branch: refs/heads/cassandra-3.X
Commit: 707c36430918b27f0c392dc0bf077af23be06c70
Parents: 55adebe
Author: Robert Stupp 
Authored: Tue Dec 13 17:37:09 2016 +0100
Committer: Robert Stupp 
Committed: Tue Dec 13 17:37:47 2016 +0100

--
 CHANGES.txt |  1 +
 .../concurrent/NamedThreadFactory.java  | 42 -
 .../functions/ThreadAwareSecurityManager.java   |  3 +-
 .../AbstractCommitLogSegmentManager.java|  2 +-
 .../db/commitlog/AbstractCommitLogService.java  |  2 +-
 .../db/compaction/CompactionManager.java|  3 +-
 .../apache/cassandra/db/marshal/AsciiType.java  |  3 +-
 .../hints/EncryptedChecksummedDataInput.java|  3 +-
 .../cassandra/index/sasi/TermIterator.java  |  5 +-
 .../io/compress/DeflateCompressor.java  | 11 +--
 .../cassandra/net/OutboundTcpConnection.java|  5 +-
 .../apache/cassandra/repair/RepairRunnable.java |  6 +-
 .../scheduler/RoundRobinScheduler.java  |  2 +-
 .../cassandra/security/CipherFactory.java   |  3 +-
 .../cassandra/security/EncryptionUtils.java |  3 +-
 .../serializers/TimestampSerializer.java|  7 +-
 .../cassandra/service/StorageService.java   |  8 +-
 .../cassandra/thrift/ThriftSessionManager.java  |  4 +-
 .../cassandra/utils/CoalescingStrategies.java   | 25 +++---
 .../org/apache/cassandra/utils/FBUtilities.java | 15 ++--
 .../org/apache/cassandra/cql3/ViewLongTest.java |  3 +-
 .../test/microbench/FastThreadExecutor.java | 65 +-
 .../test/microbench/FastThreadLocalBench.java   | 92 
 .../cassandra/cache/CacheProviderTest.java  |  3 +-
 .../cassandra/concurrent/WaitQueueTest.java |  4 +-
 .../cassandra/db/RecoveryManagerTest.java   | 22 +++--
 .../apache/cassandra/hints/HintsBufferTest.java |  3 +-
 .../io/sstable/IndexSummaryManagerTest.java |  3 +-
 .../io/sstable/SSTableRewriterTest.java |  5 +-
 .../apache/cassandra/service/RemoveTest.java| 26 +++---
 .../apache/cassandra/utils/TopKSamplerTest.java |  5 +-
 .../apache/cassandra/stress/StressServer.java   |  6 +-
 .../operations/userdefined/TokenRangeQuery.java |  3 +-
 33 files changed, 238 insertions(+), 155 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/707c3643/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a225df8..2c95d50 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.12
+ * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
  * nodetool stopdaemon errors out (CASSANDRA-13030)
  * Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028)
  * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/707c3643/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java 
b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
index 22193c4..5d89f6c 100644
--- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
@@ -20,6 +20,8 @@ package org.apache.cassandra.concurrent;
 import java.util.concurrent.ThreadFactory;
 import java.util.concurrent.atomic.AtomicInteger;
 
+import com.google.common.annotations.VisibleForTesting;
+
 import io.netty.util.concurrent.FastThreadLocal;
 import io.netty.util.concurrent.FastThreadLocalThread;
 
@@ -58,9 +60,8 @@ public class NamedThreadFactory implements ThreadFactory
 public Thread newThread(Runnable runnable)
 {
 String name = id + ':' + n.getAndIncrement();
-Thread thread = new FastThreadLocalThread(threadGroup, 
threadLocalDeallocator(runnable), name);
+Thread thread = createThread(threadGroup, runnable, name, true);
 thread.setPriority(priority);
-thread.setDaemon(true);
 if (contextClassLoader != null)
 thread.setContextClassLoader(contextClassLoader);
 return thread;
@@ -75,11 +76,44 @@ public class NamedThreadFactory implements ThreadFactory
 {
 return () ->
 {
-

[3/3] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[snazy]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d596a47c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d596a47c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d596a47c

Branch: refs/heads/trunk
Commit: d596a47cf1d2aaa8d1b4e98f886e0042a62cb9af
Parents: 4489ee8 707c364
Author: Robert Stupp 
Authored: Tue Dec 13 17:38:18 2016 +0100
Committer: Robert Stupp 
Committed: Tue Dec 13 17:38:18 2016 +0100

--
 CHANGES.txt |  1 +
 .../concurrent/NamedThreadFactory.java  | 42 -
 .../functions/ThreadAwareSecurityManager.java   |  3 +-
 .../AbstractCommitLogSegmentManager.java|  2 +-
 .../db/commitlog/AbstractCommitLogService.java  |  2 +-
 .../db/compaction/CompactionManager.java|  3 +-
 .../apache/cassandra/db/marshal/AsciiType.java  |  3 +-
 .../hints/EncryptedChecksummedDataInput.java|  3 +-
 .../cassandra/index/sasi/TermIterator.java  |  5 +-
 .../io/compress/DeflateCompressor.java  | 11 +--
 .../cassandra/net/OutboundTcpConnection.java|  5 +-
 .../apache/cassandra/repair/RepairRunnable.java |  6 +-
 .../cassandra/security/CipherFactory.java   |  3 +-
 .../cassandra/security/EncryptionUtils.java |  3 +-
 .../serializers/TimestampSerializer.java|  7 +-
 .../cassandra/service/StorageService.java   |  8 +-
 .../cassandra/utils/CoalescingStrategies.java   | 25 +++---
 .../org/apache/cassandra/utils/FBUtilities.java | 15 ++--
 .../org/apache/cassandra/cql3/ViewLongTest.java |  3 +-
 .../test/microbench/FastThreadExecutor.java | 65 +-
 .../test/microbench/FastThreadLocalBench.java   | 92 
 .../cassandra/cache/CacheProviderTest.java  |  3 +-
 .../cassandra/concurrent/WaitQueueTest.java |  4 +-
 .../cassandra/db/RecoveryManagerTest.java   | 22 +++--
 .../apache/cassandra/hints/HintsBufferTest.java |  3 +-
 .../io/sstable/IndexSummaryManagerTest.java |  3 +-
 .../io/sstable/SSTableRewriterTest.java |  5 +-
 .../apache/cassandra/service/RemoveTest.java| 26 +++---
 .../apache/cassandra/utils/TopKSamplerTest.java |  5 +-
 .../apache/cassandra/stress/StressServer.java   |  6 +-
 .../operations/userdefined/TokenRangeQuery.java |  3 +-
 31 files changed, 234 insertions(+), 153 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/CHANGES.txt
--
diff --cc CHANGES.txt
index fa81b5f,2c95d50..1f18fd9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 +4.0
 + * Thrift removal (CASSANDRA-5)
 + * Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
 + * Add column definition kind to dropped columns in schema (CASSANDRA-12705)
 + * Add (automate) Nodetool Documentation (CASSANDRA-12672)
 + * Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
 + * Reject invalid replication settings when creating or altering a keyspace 
(CASSANDRA-12681)
 + * Clean up the SSTableReader#getScanner API wrt removal of RateLimiter 
(CASSANDRA-12422)
 +
 +
  3.12
+  * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
   * nodetool stopdaemon errors out (CASSANDRA-13030)
   * Log message size in trace message in OutboundTcpConnection 
(CASSANDRA-13028)
   * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/test/long/org/apache/cassandra/cql3/ViewLongTest.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/test/unit/org/apache/cassandra/cache/CacheProviderTest.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/test/unit/org/apache/cassandra/io/sstable/SSTableRewriterTest.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d596a47c/tools/stress/src/org/apache/cassandra/stress/operations/userdefined/TokenRangeQuery.java

[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745581#comment-15745581
 ] 

Corentin Chary commented on CASSANDRA-13038:


I can confirm that the patched version still spend 30% of the compaction time 
in this code.

> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745550#comment-15745550
 ] 

Corentin Chary commented on CASSANDRA-13038:


I'll run a patched version, but according to @pcmanus CASSANDRA-13040 probably 
doesn't change much to that.

> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12510) Disallow decommission when number of replicas will drop below configured RF

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745499#comment-15745499
 ] 

Kurt Greaves commented on CASSANDRA-12510:
--

this is true... and why coding at 4am is not a good idea. I've now patched the 
other tests as well (except for the materialised views test which appears 
unrelated). I didn't get a pass myself on the resumable_decommission_test but 
it got past the decommission stage and failed at byteman (probably because of 
some misconfiguration on my end) so I think it should work.

> Disallow decommission when number of replicas will drop below configured RF
> ---
>
> Key: CASSANDRA-12510
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12510
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
> Environment: C* version 3.3
>Reporter: Atin Sood
>Assignee: Kurt Greaves
>Priority: Minor
>  Labels: lhf
> Attachments: 12510-3.x.patch
>
>
> Steps to replicate :
> - Create a 3 node cluster in DC1 and create a keyspace test_keyspace with 
> table test_table with replication strategy NetworkTopologyStrategy , DC1=3 . 
> Populate some data into this table.
> - Add 5 more nodes to this cluster, but in DC2. Also do not alter the 
> keyspace to add the new DC2 to replication (this is intentional and the 
> reason why the bug shows up). So the desc keyspace should still list 
> NetworkTopologyStrategy with DC1=3 as RF
> - As expected, this will now be a 8 node cluster with 3 nodes in DC1 and 5 in 
> DC2
> - Now start decommissioning the nodes in DC1. Note that the decommission runs 
> fine on all the 3 nodes, but since the new nodes are in DC2 and the RF for 
> keyspace is restricted to DC1, the new 5 nodes won't get any data.
> - You will now end with the 5 node cluster which has no data from the 
> decommissioned 3 nodes and hence ending up in data loss
> I do understand that this problem could have been avoided if we perform an 
> alter stmt and add DC2 replication before adding the 5 nodes. But the fact 
> that decommission ran fine on the 3 nodes on DC1 without complaining that 
> there were no nodes to stream its data seems a little discomforting. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12925) AssertionError executing authz stmt on UDF without keyspace

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745437#comment-15745437
 ] 

Carl Yeksigian commented on CASSANDRA-12925:


Code looks good, and I think I understand why we can't use the {{ClientState}}, 
but just to make sure: it's because of the way we share the permissions 
statements between resources.

That is, if we had a {{GrantFunctionPermissionsStatement}}, we could add a new 
interface {{KeyspacedStatement}} (or something) which would use the 
{{ClientState}} as in {{CFStatement}}.

> AssertionError executing authz stmt on UDF without keyspace
> ---
>
> Key: CASSANDRA-12925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Performing a {{GRANT}} or {{REVOKE}} on a user defined function requires the 
> function name to be qualified by its keyspace. Unlike {{GRANT/REVOKE}} on a 
> table, the keyspace cannot be inferred from the {{ClientState}} as it's 
> needed by the parser to either lookup the function (in 2.2), or convert the 
> function arguments from CQL types to their corresponding {{AbstractType}} (in 
> 3.0+).
> Currently, performing such a statement results in an unhandled assert error 
> and a {{ServerError}} response to the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745347#comment-15745347
 ] 

Stefan Podkowinski commented on CASSANDRA-13038:


It would be great if you could do some testing again with CASSANDRA-13040. 
There still might be performance issues during compaction of sstables with many 
distinct deletion times, but it's hard to tell by just looking at the code or 
just by doing some quick local testing.

> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745291#comment-15745291
 ] 

Corentin Chary commented on CASSANDRA-13038:


Tested with 3.9 and HEAD from two weeks ago. AFAIK the current code still runs 
StreamingHistogram for *every* cell. A cf with a default ttl and a lot of cells 
is probably the worst case.


> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745263#comment-15745263
 ] 

T Jake Luciani commented on CASSANDRA-13038:


Which version? This should be fixed in CASSANDRA-9766

> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Corentin Chary
> Attachments: compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells: 
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in 
> StreamingHistogram.update() (which is used to store the estimated tombstone 
> drop times).
> This could be caused by a huge number of different deletion times which would 
> makes the bin huge but it this histogram should be capped to 100 keys. It's 
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the 
> fact the this table has a TWCS also gives us an additional hint that sampling 
> deletion times would be fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9792) Reduce Merkle tree serialized size

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9792:
---
Assignee: Bharatendra Boddu

> Reduce Merkle tree serialized size
> --
>
> Key: CASSANDRA-9792
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9792
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Bharatendra Boddu
>Assignee: Bharatendra Boddu
>Priority: Minor
> Fix For: 3.0 beta 1
>
> Attachments: MerkleTree.java.patch, Token.java.patch
>
>
> This patch reduces the serialized size of a Merkle Tree by 10%.  With 
> num_tokens set to 256, 10% reduction in Merkle tree serialized size for each 
> token range repair, improves network bandwidth during repair 
> This table describes serialized sizes (in bytes) of Merkle trees with 
> different depths before and after patch. 
> Serialized size of a Merkle tree with certain depth, doesn't depend on number 
> of keys it represent.
> | Depth | Before patch | After patch |  Diff |
> |---+--+-+---|
> | 5 | 2060 |1840 |   220 |
> | 6 | 4044 |3600 |   444 |
> | 7 | 8012 |7120 |   892 |
> | 8 |15948 |   14160 |  1788 |
> | 9 |31820 |   28240 |  3580 |
> |10 |63564 |   56400 |  7164 |
> |11 |   127052 |  112720 | 14332 |
> |12 |   254028 |  225360 | 28668 |
> |13 |   507980 |  450640 | 57340 |
> Merkle tree with depth 15, uses serialized size of ~2MB and with this patch 
> it will be reduce the size by ~200KB. Repairing 256 token ranges will save 
> ~50MB in transfer.
> Also if token serialize() method uses, byte type to represent a token size, 
> then the serialized size can be reduced by 30 to 40%.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13040:

   Resolution: Fixed
Fix Version/s: 4.0
   3.12
   3.10
   3.0.11
   Status: Resolved  (was: Patch Available)

+1

committed to as sha b8ed255fde2cfa4a4c1cebed658f7cb817f1129d. Thanks!

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 3.0.11, 3.10, 3.12, 4.0
>
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745135#comment-15745135
 ] 

Stefan Podkowinski commented on CASSANDRA-13040:


||3.0||3.x||trunk||
|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13040-3.0]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13040-3.x]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13040-trunk]|
|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13040-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13040-3.x-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13040-trunk-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13040-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13040-3.x-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-13040-trunk-testall/]|


> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[09/10] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[jasobrown]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55adebe6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55adebe6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55adebe6

Branch: refs/heads/trunk
Commit: 55adebe665e6e1ce7f40d2cefde48b5b167b8394
Parents: ee0b743 30e154d
Author: Jason Brown 
Authored: Tue Dec 13 05:10:23 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 05:11:29 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/55adebe6/CHANGES.txt
--

[07/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30e154d3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30e154d3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30e154d3

Branch: refs/heads/trunk
Commit: 30e154d3d46b655bdd630434ca218498b16f2f41
Parents: 773dfc4 b8ed255
Author: Jason Brown 
Authored: Tue Dec 13 05:07:16 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 05:09:50 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30e154d3/CHANGES.txt
--
diff --cc CHANGES.txt
index 04d5473,38660a0..953b720
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 

[04/10] cassandra git commit: Ignore Cell.NO_DELETION_TIME for tombstone histograms

[jasobrown]

Ignore Cell.NO_DELETION_TIME for tombstone histograms

patch by Stefan Podkowinski; reviewed by jasobrown for CASSANDRA-13040


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8ed255f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8ed255f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8ed255f

Branch: refs/heads/trunk
Commit: b8ed255fde2cfa4a4c1cebed658f7cb817f1129d
Parents: 5d8f6c6
Author: Stefan Podkowinski 
Authored: Wed Dec 7 21:41:39 2016 +0100
Committer: Jason Brown 
Committed: Tue Dec 13 05:06:01 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d8bb3a..38660a0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java 
b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
index 1ff2ca8..4a67623 100644
--- a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
+++ b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
@@ -208,7 +208,8 @@ public class MetadataCollector implements 
PartitionStatisticsCollector
 private void updateLocalDeletionTime(int newLocalDeletionTime)
 {
 localDeletionTimeTracker.update(newLocalDeletionTime);
-estimatedTombstoneDropTime.update(newLocalDeletionTime);
+if (newLocalDeletionTime != Cell.NO_DELETION_TIME)
+estimatedTombstoneDropTime.update(newLocalDeletionTime);
 }
 
 private void updateTTL(int newTTL)

[06/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30e154d3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30e154d3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30e154d3

Branch: refs/heads/cassandra-3.11
Commit: 30e154d3d46b655bdd630434ca218498b16f2f41
Parents: 773dfc4 b8ed255
Author: Jason Brown 
Authored: Tue Dec 13 05:07:16 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 05:09:50 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30e154d3/CHANGES.txt
--
diff --cc CHANGES.txt
index 04d5473,38660a0..953b720
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision 

[08/10] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[jasobrown]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55adebe6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55adebe6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55adebe6

Branch: refs/heads/cassandra-3.X
Commit: 55adebe665e6e1ce7f40d2cefde48b5b167b8394
Parents: ee0b743 30e154d
Author: Jason Brown 
Authored: Tue Dec 13 05:10:23 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 05:11:29 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/55adebe6/CHANGES.txt
--

[02/10] cassandra git commit: Ignore Cell.NO_DELETION_TIME for tombstone histograms

[jasobrown]

Ignore Cell.NO_DELETION_TIME for tombstone histograms

patch by Stefan Podkowinski; reviewed by jasobrown for CASSANDRA-13040


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8ed255f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8ed255f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8ed255f

Branch: refs/heads/cassandra-3.11
Commit: b8ed255fde2cfa4a4c1cebed658f7cb817f1129d
Parents: 5d8f6c6
Author: Stefan Podkowinski 
Authored: Wed Dec 7 21:41:39 2016 +0100
Committer: Jason Brown 
Committed: Tue Dec 13 05:06:01 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d8bb3a..38660a0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java 
b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
index 1ff2ca8..4a67623 100644
--- a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
+++ b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
@@ -208,7 +208,8 @@ public class MetadataCollector implements 
PartitionStatisticsCollector
 private void updateLocalDeletionTime(int newLocalDeletionTime)
 {
 localDeletionTimeTracker.update(newLocalDeletionTime);
-estimatedTombstoneDropTime.update(newLocalDeletionTime);
+if (newLocalDeletionTime != Cell.NO_DELETION_TIME)
+estimatedTombstoneDropTime.update(newLocalDeletionTime);
 }
 
 private void updateTTL(int newTTL)

[10/10] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[jasobrown]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4489ee8d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4489ee8d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4489ee8d

Branch: refs/heads/trunk
Commit: 4489ee8d86c3af0fd78068b374e7ee414006ba52
Parents: 8b9f5e5 55adebe
Author: Jason Brown 
Authored: Tue Dec 13 05:11:58 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 05:12:35 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4489ee8d/CHANGES.txt
--
diff --cc CHANGES.txt
index 697bdbf,a225df8..fa81b5f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -132,8 -122,9 +132,9 @@@
   * Remove pre-startup check for open JMX port (CASSANDRA-12074)
   * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738)
   * Restore resumable hints delivery (CASSANDRA-11960)
 - * Properly report LWT contention (CASSANDRA-12626)
 + * Properly record CAS contention (CASSANDRA-12626)
  Merged from 3.0:
+  * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
   * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
   * Thread local pools never cleaned up (CASSANDRA-13033)
   * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)

[05/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30e154d3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30e154d3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30e154d3

Branch: refs/heads/cassandra-3.X
Commit: 30e154d3d46b655bdd630434ca218498b16f2f41
Parents: 773dfc4 b8ed255
Author: Jason Brown 
Authored: Tue Dec 13 05:07:16 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 05:09:50 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30e154d3/CHANGES.txt
--
diff --cc CHANGES.txt
index 04d5473,38660a0..953b720
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision 

[03/10] cassandra git commit: Ignore Cell.NO_DELETION_TIME for tombstone histograms

[jasobrown]

Ignore Cell.NO_DELETION_TIME for tombstone histograms

patch by Stefan Podkowinski; reviewed by jasobrown for CASSANDRA-13040


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8ed255f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8ed255f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8ed255f

Branch: refs/heads/cassandra-3.X
Commit: b8ed255fde2cfa4a4c1cebed658f7cb817f1129d
Parents: 5d8f6c6
Author: Stefan Podkowinski 
Authored: Wed Dec 7 21:41:39 2016 +0100
Committer: Jason Brown 
Committed: Tue Dec 13 05:06:01 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d8bb3a..38660a0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java 
b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
index 1ff2ca8..4a67623 100644
--- a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
+++ b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
@@ -208,7 +208,8 @@ public class MetadataCollector implements 
PartitionStatisticsCollector
 private void updateLocalDeletionTime(int newLocalDeletionTime)
 {
 localDeletionTimeTracker.update(newLocalDeletionTime);
-estimatedTombstoneDropTime.update(newLocalDeletionTime);
+if (newLocalDeletionTime != Cell.NO_DELETION_TIME)
+estimatedTombstoneDropTime.update(newLocalDeletionTime);
 }
 
 private void updateTTL(int newTTL)

[01/10] cassandra git commit: Ignore Cell.NO_DELETION_TIME for tombstone histograms

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 5d8f6c64c -> b8ed255fd
  refs/heads/cassandra-3.11 773dfc43c -> 30e154d3d
  refs/heads/cassandra-3.X ee0b74385 -> 55adebe66
  refs/heads/trunk 8b9f5e5f1 -> 4489ee8d8


Ignore Cell.NO_DELETION_TIME for tombstone histograms

patch by Stefan Podkowinski; reviewed by jasobrown for CASSANDRA-13040


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8ed255f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8ed255f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8ed255f

Branch: refs/heads/cassandra-3.0
Commit: b8ed255fde2cfa4a4c1cebed658f7cb817f1129d
Parents: 5d8f6c6
Author: Stefan Podkowinski 
Authored: Wed Dec 7 21:41:39 2016 +0100
Committer: Jason Brown 
Committed: Tue Dec 13 05:06:01 2016 -0800

--
 CHANGES.txt   | 1 +
 .../apache/cassandra/io/sstable/metadata/MetadataCollector.java   | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d8bb3a..38660a0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ed255f/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java 
b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
index 1ff2ca8..4a67623 100644
--- a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
+++ b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java
@@ -208,7 +208,8 @@ public class MetadataCollector implements 
PartitionStatisticsCollector
 private void updateLocalDeletionTime(int newLocalDeletionTime)
 {
 localDeletionTimeTracker.update(newLocalDeletionTime);
-estimatedTombstoneDropTime.update(newLocalDeletionTime);
+if (newLocalDeletionTime != Cell.NO_DELETION_TIME)
+estimatedTombstoneDropTime.update(newLocalDeletionTime);
 }
 
 private void updateTTL(int newTTL)

[jira] [Updated] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13040:

Reviewer: Jason Brown

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13021) Nodetool compactionstats fails with NullPointerException

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13021:

   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   3.0.11
   Status: Resolved  (was: Patch Available)

lgtm, +1

committed only to 3.0 as sha 5d8f6c64c40612462bbecaef8e21c0f7baf5b815

> Nodetool compactionstats fails with NullPointerException
> 
>
> Key: CASSANDRA-13021
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13021
> Project: Cassandra
>  Issue Type: Bug
> Environment: 3.0.10
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Fix For: 3.0.11
>
> Attachments: 13021-3.0-update.txt, 13021-3.0-update2.txt, 
> 13021-3.0.txt
>
>
> Found in 3.0.10:
> {code}
> $ nodetool compactionstats
> pending tasks: 2
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at 
> org.apache.cassandra.tools.nodetool.CompactionStats.addLine(CompactionStats.java:102)
> at 
> org.apache.cassandra.tools.nodetool.CompactionStats.execute(CompactionStats.java:70)
> at 
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:247)
> at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:161)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[02/10] cassandra git commit: Fix nodetool compactionstats NPE

[jasobrown]

Fix nodetool compactionstats NPE

patch by Jay Zhuang; reviewed by jasobrown for CASSANDRA-13021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d8f6c64
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d8f6c64
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d8f6c64

Branch: refs/heads/cassandra-3.11
Commit: 5d8f6c64c40612462bbecaef8e21c0f7baf5b815
Parents: 36ce4e0
Author: Jay Zhuang 
Authored: Thu Dec 8 14:56:18 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:32:34 2016 -0800

--
 CHANGES.txt  |  1 +
 .../cassandra/tools/nodetool/CompactionStats.java| 15 +--
 2 files changed, 10 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a65a147..4d8bb3a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
  * CQL often queries static columns unnecessarily (CASSANDRA-12768)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
index e57d2ee..bac23c2 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
@@ -30,7 +30,6 @@ import 
org.apache.cassandra.db.compaction.CompactionManagerMBean;
 import org.apache.cassandra.db.compaction.OperationType;
 import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.tools.NodeProbe;
-import org.apache.cassandra.tools.NodeTool;
 import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 
 @Command(name = "compactionstats", description = "Print statistics on 
compactions")
@@ -96,10 +95,14 @@ public class CompactionStats extends NodeToolCmd
 }
 }
 
-private void addLine(List lines, int[] columnSizes, String... 
columns) {
-lines.add(columns);
-for (int i = 0; i < columns.length; i++) {
-columnSizes[i] = Math.max(columnSizes[i], columns[i].length());
+private void addLine(List lines, int[] columnSizes, String... 
columns)
+{
+String[] newColumns = new String[columns.length];
+for (int i = 0; i < columns.length; i++)
+{
+columnSizes[i] = Math.max(columnSizes[i], columns[i] != null ? 
columns[i].length() : 1);
+newColumns[i] = columns[i] != null ? columns[i] : "";
 }
+lines.add(newColumns);
 }
-}
\ No newline at end of file
+}

[03/10] cassandra git commit: Fix nodetool compactionstats NPE

[jasobrown]

Fix nodetool compactionstats NPE

patch by Jay Zhuang; reviewed by jasobrown for CASSANDRA-13021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d8f6c64
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d8f6c64
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d8f6c64

Branch: refs/heads/cassandra-3.X
Commit: 5d8f6c64c40612462bbecaef8e21c0f7baf5b815
Parents: 36ce4e0
Author: Jay Zhuang 
Authored: Thu Dec 8 14:56:18 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:32:34 2016 -0800

--
 CHANGES.txt  |  1 +
 .../cassandra/tools/nodetool/CompactionStats.java| 15 +--
 2 files changed, 10 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a65a147..4d8bb3a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
  * CQL often queries static columns unnecessarily (CASSANDRA-12768)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
index e57d2ee..bac23c2 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
@@ -30,7 +30,6 @@ import 
org.apache.cassandra.db.compaction.CompactionManagerMBean;
 import org.apache.cassandra.db.compaction.OperationType;
 import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.tools.NodeProbe;
-import org.apache.cassandra.tools.NodeTool;
 import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 
 @Command(name = "compactionstats", description = "Print statistics on 
compactions")
@@ -96,10 +95,14 @@ public class CompactionStats extends NodeToolCmd
 }
 }
 
-private void addLine(List lines, int[] columnSizes, String... 
columns) {
-lines.add(columns);
-for (int i = 0; i < columns.length; i++) {
-columnSizes[i] = Math.max(columnSizes[i], columns[i].length());
+private void addLine(List lines, int[] columnSizes, String... 
columns)
+{
+String[] newColumns = new String[columns.length];
+for (int i = 0; i < columns.length; i++)
+{
+columnSizes[i] = Math.max(columnSizes[i], columns[i] != null ? 
columns[i].length() : 1);
+newColumns[i] = columns[i] != null ? columns[i] : "";
 }
+lines.add(newColumns);
 }
-}
\ No newline at end of file
+}

[07/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/773dfc43
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/773dfc43
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/773dfc43

Branch: refs/heads/trunk
Commit: 773dfc43c911240a7de457101ee2f458442ba943
Parents: 8bbe2f5 5d8f6c6
Author: Jason Brown 
Authored: Tue Dec 13 04:33:19 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:34:39 2016 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/773dfc43/CHANGES.txt
--
diff --cc CHANGES.txt
index 2156ab9,4d8bb3a..04d5473
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to streaming 

[08/10] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[jasobrown]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ee0b7438
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ee0b7438
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ee0b7438

Branch: refs/heads/cassandra-3.X
Commit: ee0b743859c2282f5a6c0c94fb6e4c8983452fd7
Parents: 6459f5a 773dfc4
Author: Jason Brown 
Authored: Tue Dec 13 04:34:59 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:35:31 2016 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee0b7438/CHANGES.txt
--

[06/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/773dfc43
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/773dfc43
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/773dfc43

Branch: refs/heads/cassandra-3.11
Commit: 773dfc43c911240a7de457101ee2f458442ba943
Parents: 8bbe2f5 5d8f6c6
Author: Jason Brown 
Authored: Tue Dec 13 04:33:19 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:34:39 2016 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/773dfc43/CHANGES.txt
--
diff --cc CHANGES.txt
index 2156ab9,4d8bb3a..04d5473
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to 

[01/10] cassandra git commit: Fix nodetool compactionstats NPE

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 36ce4e02b -> 5d8f6c64c
  refs/heads/cassandra-3.11 8bbe2f595 -> 773dfc43c
  refs/heads/cassandra-3.X 6459f5ab1 -> ee0b74385
  refs/heads/trunk bb20b9dd8 -> 8b9f5e5f1


Fix nodetool compactionstats NPE

patch by Jay Zhuang; reviewed by jasobrown for CASSANDRA-13021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d8f6c64
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d8f6c64
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d8f6c64

Branch: refs/heads/cassandra-3.0
Commit: 5d8f6c64c40612462bbecaef8e21c0f7baf5b815
Parents: 36ce4e0
Author: Jay Zhuang 
Authored: Thu Dec 8 14:56:18 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:32:34 2016 -0800

--
 CHANGES.txt  |  1 +
 .../cassandra/tools/nodetool/CompactionStats.java| 15 +--
 2 files changed, 10 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a65a147..4d8bb3a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
  * CQL often queries static columns unnecessarily (CASSANDRA-12768)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
index e57d2ee..bac23c2 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
@@ -30,7 +30,6 @@ import 
org.apache.cassandra.db.compaction.CompactionManagerMBean;
 import org.apache.cassandra.db.compaction.OperationType;
 import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.tools.NodeProbe;
-import org.apache.cassandra.tools.NodeTool;
 import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 
 @Command(name = "compactionstats", description = "Print statistics on 
compactions")
@@ -96,10 +95,14 @@ public class CompactionStats extends NodeToolCmd
 }
 }
 
-private void addLine(List lines, int[] columnSizes, String... 
columns) {
-lines.add(columns);
-for (int i = 0; i < columns.length; i++) {
-columnSizes[i] = Math.max(columnSizes[i], columns[i].length());
+private void addLine(List lines, int[] columnSizes, String... 
columns)
+{
+String[] newColumns = new String[columns.length];
+for (int i = 0; i < columns.length; i++)
+{
+columnSizes[i] = Math.max(columnSizes[i], columns[i] != null ? 
columns[i].length() : 1);
+newColumns[i] = columns[i] != null ? columns[i] : "";
 }
+lines.add(newColumns);
 }
-}
\ No newline at end of file
+}

[09/10] cassandra git commit: Merge branch 'cassandra-3.11' into cassandra-3.X

[jasobrown]

Merge branch 'cassandra-3.11' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ee0b7438
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ee0b7438
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ee0b7438

Branch: refs/heads/trunk
Commit: ee0b743859c2282f5a6c0c94fb6e4c8983452fd7
Parents: 6459f5a 773dfc4
Author: Jason Brown 
Authored: Tue Dec 13 04:34:59 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:35:31 2016 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ee0b7438/CHANGES.txt
--

[10/10] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[jasobrown]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8b9f5e5f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8b9f5e5f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8b9f5e5f

Branch: refs/heads/trunk
Commit: 8b9f5e5f189fe291167e33f956a462986b841e4f
Parents: bb20b9d ee0b743
Author: Jason Brown 
Authored: Tue Dec 13 04:36:26 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:36:48 2016 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b9f5e5f/CHANGES.txt
--
diff --cc CHANGES.txt
index 3a7818e,e3cd5a1..697bdbf
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -132,8 -122,9 +132,9 @@@
   * Remove pre-startup check for open JMX port (CASSANDRA-12074)
   * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738)
   * Restore resumable hints delivery (CASSANDRA-11960)
 - * Properly report LWT contention (CASSANDRA-12626)
 + * Properly record CAS contention (CASSANDRA-12626)
  Merged from 3.0:
+  * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
   * Thread local pools never cleaned up (CASSANDRA-13033)
   * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
   * CQL often queries static columns unnecessarily (CASSANDRA-12768)

[05/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/773dfc43
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/773dfc43
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/773dfc43

Branch: refs/heads/cassandra-3.X
Commit: 773dfc43c911240a7de457101ee2f458442ba943
Parents: 8bbe2f5 5d8f6c6
Author: Jason Brown 
Authored: Tue Dec 13 04:33:19 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:34:39 2016 -0800

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/773dfc43/CHANGES.txt
--
diff --cc CHANGES.txt
index 2156ab9,4d8bb3a..04d5473
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,114 -1,5 +1,115 @@@
 -3.0.11
 +3.10
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to 

[04/10] cassandra git commit: Fix nodetool compactionstats NPE

[jasobrown]

Fix nodetool compactionstats NPE

patch by Jay Zhuang; reviewed by jasobrown for CASSANDRA-13021


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d8f6c64
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d8f6c64
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d8f6c64

Branch: refs/heads/trunk
Commit: 5d8f6c64c40612462bbecaef8e21c0f7baf5b815
Parents: 36ce4e0
Author: Jay Zhuang 
Authored: Thu Dec 8 14:56:18 2016 -0800
Committer: Jason Brown 
Committed: Tue Dec 13 04:32:34 2016 -0800

--
 CHANGES.txt  |  1 +
 .../cassandra/tools/nodetool/CompactionStats.java| 15 +--
 2 files changed, 10 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a65a147..4d8bb3a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
  * Thread local pools never cleaned up (CASSANDRA-13033)
  * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
  * CQL often queries static columns unnecessarily (CASSANDRA-12768)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d8f6c64/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
index e57d2ee..bac23c2 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/CompactionStats.java
@@ -30,7 +30,6 @@ import 
org.apache.cassandra.db.compaction.CompactionManagerMBean;
 import org.apache.cassandra.db.compaction.OperationType;
 import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.tools.NodeProbe;
-import org.apache.cassandra.tools.NodeTool;
 import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 
 @Command(name = "compactionstats", description = "Print statistics on 
compactions")
@@ -96,10 +95,14 @@ public class CompactionStats extends NodeToolCmd
 }
 }
 
-private void addLine(List lines, int[] columnSizes, String... 
columns) {
-lines.add(columns);
-for (int i = 0; i < columns.length; i++) {
-columnSizes[i] = Math.max(columnSizes[i], columns[i].length());
+private void addLine(List lines, int[] columnSizes, String... 
columns)
+{
+String[] newColumns = new String[columns.length];
+for (int i = 0; i < columns.length; i++)
+{
+columnSizes[i] = Math.max(columnSizes[i], columns[i] != null ? 
columns[i].length() : 1);
+newColumns[i] = columns[i] != null ? columns[i] : "";
 }
+lines.add(newColumns);
 }
-}
\ No newline at end of file
+}

[jira] [Updated] (CASSANDRA-13034) Move to FastThreadLocalThread and FastThreadLocal

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13034:

Status: Ready to Commit  (was: Patch Available)

> Move to FastThreadLocalThread and FastThreadLocal
> -
>
> Key: CASSANDRA-13034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13034
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> (Supersedes/includes CASSANDRA-13033 for 3.X & trunk)
> We still use {{ThreadLocal}} in a couple of places, so I was curious how much 
> faster {{FastThreadLocal}} is compared to {{ThreadLocal}}. A micro bench 
> tells, that {{FastThreadLocal}} has a runtime of ~2.7ns and {{ThreadLocal}} 
> of ~4.7ns - about 2ns slower (EDIT: subtracted baseline).
> However, looking at the implementations it seems that {{ThreadLocal}} has 
> more dependent pointer gets than {{FastThreadLocal}}. This (CPU cache misses) 
> is not reflected in the artificial benchmark below.
> The patch migrates all {{Thread}} instances (except a few in tests) and all 
> {{ThreadLocal}} instances.
> {code:title=FastThreadLocalBench with 4 threads on 4 core CPU}
>  [java] FastThreadLocalBench.baseline   2  avgt5  
> 3.023 ± 0.081  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal2  avgt5  
> 5.610 ± 0.154  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal4  avgt5  
> 5.653 ± 0.042  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal8  avgt5  
> 5.763 ± 0.588  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal   12  avgt5  
> 5.673 ± 0.117  ns/op
>  [java] FastThreadLocalBench.threadLocal2  avgt5  
> 7.708 ± 0.723  ns/op
>  [java] FastThreadLocalBench.threadLocal4  avgt5  
> 7.604 ± 0.059  ns/op
>  [java] FastThreadLocalBench.threadLocal8  avgt5  
> 7.629 ± 0.080  ns/op
>  [java] FastThreadLocalBench.threadLocal   12  avgt5  
> 7.858 ± 0.483  ns/op
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13034) Move to FastThreadLocalThread and FastThreadLocal

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13034:

Reviewer: Jason Brown

> Move to FastThreadLocalThread and FastThreadLocal
> -
>
> Key: CASSANDRA-13034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13034
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> (Supersedes/includes CASSANDRA-13033 for 3.X & trunk)
> We still use {{ThreadLocal}} in a couple of places, so I was curious how much 
> faster {{FastThreadLocal}} is compared to {{ThreadLocal}}. A micro bench 
> tells, that {{FastThreadLocal}} has a runtime of ~2.7ns and {{ThreadLocal}} 
> of ~4.7ns - about 2ns slower (EDIT: subtracted baseline).
> However, looking at the implementations it seems that {{ThreadLocal}} has 
> more dependent pointer gets than {{FastThreadLocal}}. This (CPU cache misses) 
> is not reflected in the artificial benchmark below.
> The patch migrates all {{Thread}} instances (except a few in tests) and all 
> {{ThreadLocal}} instances.
> {code:title=FastThreadLocalBench with 4 threads on 4 core CPU}
>  [java] FastThreadLocalBench.baseline   2  avgt5  
> 3.023 ± 0.081  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal2  avgt5  
> 5.610 ± 0.154  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal4  avgt5  
> 5.653 ± 0.042  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal8  avgt5  
> 5.763 ± 0.588  ns/op
>  [java] FastThreadLocalBench.fastThreadLocal   12  avgt5  
> 5.673 ± 0.117  ns/op
>  [java] FastThreadLocalBench.threadLocal2  avgt5  
> 7.708 ± 0.723  ns/op
>  [java] FastThreadLocalBench.threadLocal4  avgt5  
> 7.604 ± 0.059  ns/op
>  [java] FastThreadLocalBench.threadLocal8  avgt5  
> 7.629 ± 0.080  ns/op
>  [java] FastThreadLocalBench.threadLocal   12  avgt5  
> 7.858 ± 0.483  ns/op
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12959) copy from csv import wrong values with udt having set when fields are not specified in correct order in csv

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12959:

Reproduced In: 3.9, 3.8  (was: 3.8, 3.9)
 Reviewer: Paulo Motta

> copy from csv import wrong values with udt having set when fields are not 
> specified in correct order in csv
> ---
>
> Key: CASSANDRA-12959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12959
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Quentin Ambard
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
>  {quote}
> create KEYSPACE test WITH replication = \{ 'class': 'SimpleStrategy', 
> 'replication_factor': 1\};
> CREATE TYPE test.my_udt (
> first_field text,
> second_field frozen
> );
> CREATE TABLE test.test ( key text, value my_udt, PRIMARY KEY (key));
> {quote}
> The following works as expected : 
> {quote}
> INSERT INTO test.test (key , value ) VALUES ( 'key1', \{second_field: 
> \{'test1', 'test2'\}, first_field: 'first_field'\});
>  key  | value
> --+
>  key1 | \{first_field: 'first_field', second_field: \{'test1', 'test2'\}\}
> {quote}
> but when inserted using a .csv the result is wrong:
> {quote}"key1","\{second_field: \{'test1', 'test2'\}, first_field: 
> 'first_field'\}"
> COPY test.test FROM '~/test.csv';
>  key  | value
> --+--
>  key1 | \{first_field: '\{''test1'', ''test2''\}', second_field: 
> \{'irst_fiel'\}\}
> {quote}
> it works as expected if the keys are in order: 
> bq. "key1","\{first_field: 'first_field', second_field: \{'test1', 
> 'test2'\}\}")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Ninja: fix ant javadoc target

[samt]

Repository: cassandra
Updated Branches:
  refs/heads/trunk b02dec3da -> bb20b9dd8


Ninja: fix ant javadoc target


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb20b9dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb20b9dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb20b9dd

Branch: refs/heads/trunk
Commit: bb20b9dd8c3a030b2055438566a362f8e60fb76c
Parents: b02dec3
Author: Sam Tunnicliffe 
Authored: Tue Dec 13 11:14:36 2016 +
Committer: Sam Tunnicliffe 
Committed: Tue Dec 13 11:14:36 2016 +

--
 build.xml | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bb20b9dd/build.xml
--
diff --git a/build.xml b/build.xml
index d1ffb7a..06a7938 100644
--- a/build.xml
+++ b/build.xml
@@ -1456,9 +1456,11 @@
 
   
 
-  
-
-  
+  
+
+  
+
+  
 

 

[jira] [Updated] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13040:
---
Attachment: (was: 13040-3.0.patch)

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13040:
---
Attachment: 13040-3.0.patch

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13040:
---
Attachment: 13040-3.0.patch

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: 13040-3.0.patch
>
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13040:
---
Status: Patch Available  (was: Open)

> Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME
> ---
>
> Key: CASSANDRA-13040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>
> Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
> newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
> order to update the min/max deletion times as well as the 
> {{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
> {{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
> to be called for every regular cell.
> This can be easily verified by using {{sstablemetadata}} on any sstable. You 
> should be able to find a very high value for 2147483647 in the "Estimated 
> tombstone drop times" section.
> At this point, I don't think this is causing serious harm, but could be a 
> performance issue (see linked ticket). Calculated droppable TS times should 
> not be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13040) Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME

[Stefan Podkowinski (JIRA)]

Stefan Podkowinski created CASSANDRA-13040:
--

 Summary: Estimated TS drop-time histogram updated with 
Cell.NO_DELETION_TIME
 Key: CASSANDRA-13040
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13040
 Project: Cassandra
  Issue Type: Bug
Reporter: Stefan Podkowinski
Assignee: Stefan Podkowinski


Starting with 3.0, {{MetadataCollector.updateLocalDeletionTime(int 
newLocalDeletionTime)}} will be called with {{cell.localDeletionTime()}} in 
order to update the min/max deletion times as well as the 
{{estimatedTombstoneDropTime}} histogram. Currently this also happens for 
{{Cell.NO_DELETION_TIME}} ({{Integer.MAX_VALUE}}), which causes the histogram 
to be called for every regular cell.

This can be easily verified by using {{sstablemetadata}} on any sstable. You 
should be able to find a very high value for 2147483647 in the "Estimated 
tombstone drop times" section.

At this point, I don't think this is causing serious harm, but could be a 
performance issue (see linked ticket). Calculated droppable TS times should not 
be affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)