[jira] [Commented] (CASSANDRA-11738) Re-think the use of Severity in the DynamicEndpointSnitch calculation

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290389#comment-15290389
 ] 

Jonathan Ellis commented on CASSANDRA-11738:


It could, but how would you decide when to use "load" and when to use directly 
measured latency?

> Re-think the use of Severity in the DynamicEndpointSnitch calculation
> -
>
> Key: CASSANDRA-11738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11738
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jeremiah Jordan
> Fix For: 3.x
>
>
> CASSANDRA-11737 was opened to allow completely disabling the use of severity 
> in the DynamicEndpointSnitch calculation, but that is a pretty big hammer.  
> There is probably something we can do to better use the score.
> The issue seems to be that severity is given equal weight with latency in the 
> current code, also that severity is only based on disk io.  If you have a 
> node that is CPU bound on something (say catching up on LCS compactions 
> because of bootstrap/repair/replace) the IO wait can be low, but the latency 
> to the node is high.
> Some ideas I had are:
> 1. Allowing a yaml parameter to tune how much impact the severity score has 
> in the calculation.
> 2. Taking CPU load into account as well as IO Wait (this would probably help 
> in the cases I have seen things go sideways)
> 3. Move the -D from CASSANDRA-11737 to being a yaml level setting
> 4. Go back to just relying on Latency and get rid of severity all together.  
> Now that we have rapid read protection, maybe just using latency is enough, 
> as it can help where the predictive nature of IO wait would have been useful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290375#comment-15290375
 ] 

Stefania commented on CASSANDRA-11055:
--

Thanks for opening 
[PYTHON-562|https://datastax-oss.atlassian.net/browse/PYTHON-562] and for the 
refresher on how to roll with a commit included, definitely most appreciated! :)

I've created a branch for 2.1 with the updated driver and launched our own 
tests:
|[patch|https://github.com/stef1927/cassandra/commits/11055-cqlsh-2.1]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11055-cqlsh-2.1-cqlsh-tests/]|

I had a look at the legacy metadata failing test, I think it fails on the 2.7.2 
tag as well. The clustering columns come from the comparator fields and are 
sorted by name it seems. So I think it's just a matter of rearranging the order 
of clustering columns in the test but I'll let you double check.

We can resume once you are back, enjoy the conference!


> C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no 
> attribute 'replace'
> ---
>
> Key: CASSANDRA-11055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11055
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Simon Ashley
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: 11055-driver-2.7.2.patch, data.tar.gz
>
>
> C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns:
>  'NoneType' object has no attribute 'replace' 
> for thrift CF's originally created in C* 1.2.
> Repro:
> 1. Create cf in cassandra-cli on C* 1.2.x  (1.2.9 was used here)
> [default@ks1] CREATE COLUMN FAMILY t1
> ...   WITH column_type='Standard'
> ...   AND 
> comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> ...   AND default_validation_class='UTF8Type'
> ...   AND key_validation_class='UTF8Type'
> ...   AND read_repair_chance=0.1
> ...   AND dclocal_read_repair_chance=0.0
> ...   AND gc_grace=864000
> ...   AND min_compaction_threshold=4
> ...   AND max_compaction_threshold=32
> ...   AND replicate_on_write=true
> ...   AND compaction_strategy='LeveledCompactionStrategy' AND 
> compaction_strategy_options={sstable_size_in_mb: 32}
> ...   AND caching='KEYS_ONLY'
> ...   AND compression_options={sstable_compression:SnappyCompressor, 
> chunk_length_kb:64};
> qlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {
>   'class': 'NetworkTopologyStrategy',
>   'datacenter1': '1'
> };
> USE ks1;
> CREATE TABLE t1 (
>   key text,
>   column1 text,
>   column2 text,
>   value text,
>   PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE AND
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=864000 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '32', 'class': 
> 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '64', 'sstable_compression': 
> 'SnappyCompressor'};
> cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases 
> from system.schema_columnfamilies where keyspace_name= 'ks1';
>  keyspace_name | columnfamily_name | column_aliases | key_aliases
> ---+---++-
>ks1 |t1 | [] |  []
> 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a
> At this stage , DESCRIBE in cqlsh is working
> 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a
> DESCRIBE now fails:
> cqlsh> describe table ks1.t1;
> 'NoneType' object has no attribute 'replace'
> cqlsh> describe keyspace ks1;
> 'NoneType' object has no attribute 'replace'
> You can workaround by manually updating system.schema_columnfamilies
>  UPDATE system.schema_columnfamilies SET column_aliases 
> ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = 
> 't1';
> Once you exit and restart cqlsh, DESCRIBE is not working as per C* 1.2
> cqlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1'}  AND durable_writes = true;
> CREATE TABLE ks1.t1 (
> key text,
> column1 text,
> column2 text,
> value text,
> PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE
> AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'sstable_size_in_mb': '32', 'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = 

[jira] [Commented] (CASSANDRA-11569) Track message latency across DCs

[stone (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290374#comment-15290374
 ] 

stone commented on CASSANDRA-11569:
---

see nodeLatency.PNG,I change latency from 90ms to 30ms at 10:50,and workload 
are same as usual.
but I cannot see the value changes.

> Track message latency across DCs
> 
>
> Key: CASSANDRA-11569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11569
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11569.patch, CASSANDRA-11569v2.txt, 
> nodeLatency.PNG
>
>
> Since we have the timestamp a message is created and when arrives, we can get 
> an approximate time it took relatively easy and would remove necessity for 
> more complex hacks to determine latency between DCs.
> Although is not going to be very meaningful when ntp is not setup, it is 
> pretty common to have NTP setup and even with clock drift nothing is really 
> hurt except the metric becoming whacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11569) Track message latency across DCs

[stone (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stone updated CASSANDRA-11569:
--
Attachment: nodeLatency.PNG

> Track message latency across DCs
> 
>
> Key: CASSANDRA-11569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11569
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11569.patch, CASSANDRA-11569v2.txt, 
> nodeLatency.PNG
>
>
> Since we have the timestamp a message is created and when arrives, we can get 
> an approximate time it took relatively easy and would remove necessity for 
> more complex hacks to determine latency between DCs.
> Although is not going to be very meaningful when ntp is not setup, it is 
> pretty common to have NTP setup and even with clock drift nothing is really 
> hurt except the metric becoming whacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11569) Track message latency across DCs

[stone (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290370#comment-15290370
 ] 

stone commented on CASSANDRA-11569:
---

thx,what I mean is the value of crossNodeLatency is  not right.
when the cross DataCenter latency increase,the value should increase right now,
but there is a slow increase,and seems the value is not right


> Track message latency across DCs
> 
>
> Key: CASSANDRA-11569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11569
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11569.patch, CASSANDRA-11569v2.txt
>
>
> Since we have the timestamp a message is created and when arrives, we can get 
> an approximate time it took relatively easy and would remove necessity for 
> more complex hacks to determine latency between DCs.
> Although is not going to be very meaningful when ntp is not setup, it is 
> pretty common to have NTP setup and even with clock drift nothing is really 
> hurt except the metric becoming whacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11503) Need a tool to detect what percentage of SSTables on a node have been repaired when using incremental repairs.

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290359#comment-15290359
 ] 

Chris Lohfink commented on CASSANDRA-11503:
---

Added it to each table, and a global one. The per tables show up in {{nodetool 
tablestats}} and the global in {{nodetool info}}

> Need a tool to detect what percentage of SSTables on a node have been 
> repaired when using incremental repairs.
> --
>
> Key: CASSANDRA-11503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11503
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Sean Usher
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11503.patch, CASSANDRA-11503v2.patch
>
>
> When using incremental repair, we should be able to look at SSTables and 
> understand how many sstables are in the repaired and unrepaired buckets on 
> each machine. This can help us track the repair progress and if we are 
> hitting any issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11503) Need a tool to detect what percentage of SSTables on a node have been repaired when using incremental repairs.

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-11503:
--
Attachment: CASSANDRA-11503v2.patch

> Need a tool to detect what percentage of SSTables on a node have been 
> repaired when using incremental repairs.
> --
>
> Key: CASSANDRA-11503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11503
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Sean Usher
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11503.patch, CASSANDRA-11503v2.patch
>
>
> When using incremental repair, we should be able to look at SSTables and 
> understand how many sstables are in the repaired and unrepaired buckets on 
> each machine. This can help us track the repair progress and if we are 
> hitting any issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11272) NullPointerException (NPE) during bootstrap startup in StorageService.java

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290293#comment-15290293
 ] 

Joel Knighton commented on CASSANDRA-11272:
---

[~ifesdjeen] - why log e.getCause() instead of just e in your patches? No need 
to retrigger CI, but I think we should just log the original error.

> NullPointerException (NPE) during bootstrap startup in StorageService.java
> --
>
> Key: CASSANDRA-11272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11272
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
> Environment: debian jesse up to date
>Reporter: Jason Kania
>Assignee: Alex Petrov
> Fix For: 2.2.x, 3.0.x, 3.7
>
>
> After bootstrapping fails due to stream closed error, the following error 
> results:
> {code}
> Feb 27, 2016 8:06:38 PM com.google.common.util.concurrent.ExecutionList 
> executeListener
> SEVERE: RuntimeException while executing runnable 
> com.google.common.util.concurrent.Futures$6@3d61813b with executor INSTANCE
> java.lang.NullPointerException
> at 
> org.apache.cassandra.service.StorageService$2.onFailure(StorageService.java:1284)
> at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
> at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
> at 
> com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
> at 
> com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
> at 
> com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
> at 
> org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:210)
> at 
> org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:186)
> at 
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:430)
> at 
> org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:525)
> at 
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:645)
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:70)
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11557) Download page must not link to snapshots / nightly builds

[Sebb (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebb resolved CASSANDRA-11557.
--
Resolution: Fixed

Thought I could move the issue; it appears not

> Download page must not link to snapshots / nightly builds
> -
>
> Key: CASSANDRA-11557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11557
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sebb
>
> Nightly builds / snapshots which are not formal releases should not be linked 
> from the main download page.
> http://ode.apache.org/getting-ode.html#nightly-builds
> Such builds have not been voted on and should only be used by developers who 
> should be made aware that the code is without any guarantees,
> Nightly builds are not formal ASF releases, and must not be promoted to the 
> general public.
> See [1] second para. The second sentence states:
> "Do not include any links on the project website that might encourage 
> non-developers to download and use nightly builds, snapshots, release 
> candidates, or any other similar package."
> [1] http://www.apache.org/dev/release.html#what



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-11557) Download page must not link to snapshots / nightly builds

[Sebb (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebb reopened CASSANDRA-11557:
--

> Download page must not link to snapshots / nightly builds
> -
>
> Key: CASSANDRA-11557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11557
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sebb
>
> Nightly builds / snapshots which are not formal releases should not be linked 
> from the main download page.
> http://ode.apache.org/getting-ode.html#nightly-builds
> Such builds have not been voted on and should only be used by developers who 
> should be made aware that the code is without any guarantees,
> Nightly builds are not formal ASF releases, and must not be promoted to the 
> general public.
> See [1] second para. The second sentence states:
> "Do not include any links on the project website that might encourage 
> non-developers to download and use nightly builds, snapshots, release 
> candidates, or any other similar package."
> [1] http://www.apache.org/dev/release.html#what



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11731) dtest failure in pushed_notifications_test.TestPushedNotifications.move_single_node_test

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290243#comment-15290243
 ] 

Stefania commented on CASSANDRA-11731:
--

I think this line here:

{code}
waiters[0].wait_for_notifications(timeout=60, num_notifications=1)
{code}

should be:

{code}
waiters[0].wait_for_notifications(timeout=60, num_notifications=3)
{code}

since we can receive up to 3 NEW_NODE notifications.

Let's see if this fixes it, if it doesn't we need to add log messages server 
side.

> dtest failure in 
> pushed_notifications_test.TestPushedNotifications.move_single_node_test
> 
>
> Key: CASSANDRA-11731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11731
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> one recent failure (no vnode job)
> {noformat}
> 'MOVED_NODE' != u'NEW_NODE'
> {noformat}
> http://cassci.datastax.com/job/trunk_novnode_dtest/366/testReport/pushed_notifications_test/TestPushedNotifications/move_single_node_test
> Failed on CassCI build trunk_novnode_dtest #366



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11191) Support more flexible offline access to SSTable

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290190#comment-15290190
 ] 

Yuki Morishita commented on CASSANDRA-11191:


My initial approach will be to create SSTable reader/writer factory that holds 
{{Directories}}, {{CFMetaData}}(schema), and SSTable configurations. This is 
basically take some SSTable related methods and params from 
{{ColumnFamilyStore}} and move them to own class. This way, 
{{ColumnFamilyStore}} and future SSTable tools can both use this class, and 
tools do not need to access unnecessary things.

> Support more flexible offline access to SSTable
> ---
>
> Key: CASSANDRA-11191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11191
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>
> Right now, using SSTableReader/SSTableWriter alone needs certain set ups like 
> {{Config.setClientMode(true)}} or loading schema outside. Even doing so, 
> various options to read/write SSTable offline is fixed and not 
> programatically modifiable.
> We should decouple most of {{org.apache.cassandra.io.sstable}} from other 
> parts of cassandra so that we can easily reuse them in offilne tools.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290104#comment-15290104
 ] 

Adam Holmberg edited comment on CASSANDRA-11055 at 5/18/16 11:47 PM:
-

I did miss this. Thanks.

bq. Due to the API changes in the 3.0+ driver, I would prefer to apply this 
patch to the 2.7.2 branch rather than upgrade the bundled driver in 2.1 to 3.0
I agree it would be most straightforward to apply this after the 2.7.2 tag and 
package for integration. 
[Here|https://github.com/datastax/python-driver/compare/2.7.2-cassandra-2.1] is 
a branch with your patch applied 
([refresher|https://github.com/datastax/python-driver/blob/master/README-dev.rst#packaging-for-cassandra]
 on how to roll with commit included). It looks like this change breaks [one of 
our (contrived) legacy metadata 
tests|https://github.com/datastax/python-driver/blob/2.7.2-cassandra-2.1/tests/integration/standard/test_metadata.py#L869-L1133],
 so we'll have to decide if the test or the change is wrong. I'm out of office 
W-F this week so I'm not sure I'll find the time to work on it.

I will also take a look to see if this schema impacts legacy schema parsing in 
recent versions of the driver connecting to older versions of Cassandra. That 
should not be a blocker for this ticket though. Follow-on for the driver: 
[PYTHON-562|https://datastax-oss.atlassian.net/browse/PYTHON-562]


was (Author: aholmber):
I did miss this. Thanks.

bq. Due to the API changes in the 3.0+ driver, I would prefer to apply this 
patch to the 2.7.2 branch rather than upgrade the bundled driver in 2.1 to 3.0
I agree it would be most straightforward to apply this after the 2.7.2 tag and 
package for integration. 
[Here|https://github.com/datastax/python-driver/compare/2.7.2-cassandra-2.1] is 
a branch with your patch applied 
([refresher|https://github.com/datastax/python-driver/blob/master/README-dev.rst#packaging-for-cassandra]
 on how to roll with commit included). It looks like this change breaks [one of 
our *contrived* legacy metadata 
tests|https://github.com/datastax/python-driver/blob/2.7.2-cassandra-2.1/tests/integration/standard/test_metadata.py#L869-L1133],
 so we'll have to decide if the test of the change is wrong. I'm out of office 
W-F this week so I'm not sure I'll find the time to work on it.

I will also take a look to see if this schema impacts legacy schema parsing in 
recent versions of the driver connecting to older versions of Cassandra. That 
should not be a blocker for this ticket though. Follow-on for the driver: 
[PYTHON-562|https://datastax-oss.atlassian.net/browse/PYTHON-562]

> C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no 
> attribute 'replace'
> ---
>
> Key: CASSANDRA-11055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11055
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Simon Ashley
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: 11055-driver-2.7.2.patch, data.tar.gz
>
>
> C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns:
>  'NoneType' object has no attribute 'replace' 
> for thrift CF's originally created in C* 1.2.
> Repro:
> 1. Create cf in cassandra-cli on C* 1.2.x  (1.2.9 was used here)
> [default@ks1] CREATE COLUMN FAMILY t1
> ...   WITH column_type='Standard'
> ...   AND 
> comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> ...   AND default_validation_class='UTF8Type'
> ...   AND key_validation_class='UTF8Type'
> ...   AND read_repair_chance=0.1
> ...   AND dclocal_read_repair_chance=0.0
> ...   AND gc_grace=864000
> ...   AND min_compaction_threshold=4
> ...   AND max_compaction_threshold=32
> ...   AND replicate_on_write=true
> ...   AND compaction_strategy='LeveledCompactionStrategy' AND 
> compaction_strategy_options={sstable_size_in_mb: 32}
> ...   AND caching='KEYS_ONLY'
> ...   AND compression_options={sstable_compression:SnappyCompressor, 
> chunk_length_kb:64};
> qlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {
>   'class': 'NetworkTopologyStrategy',
>   'datacenter1': '1'
> };
> USE ks1;
> CREATE TABLE t1 (
>   key text,
>   column1 text,
>   column2 text,
>   value text,
>   PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE AND
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=864000 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '32', 'class': 
> 'LeveledCompactionStrategy'} AND
>   

[jira] [Commented] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290104#comment-15290104
 ] 

Adam Holmberg commented on CASSANDRA-11055:
---

I did miss this. Thanks.

bq. Due to the API changes in the 3.0+ driver, I would prefer to apply this 
patch to the 2.7.2 branch rather than upgrade the bundled driver in 2.1 to 3.0
I agree it would be most straightforward to apply this after the 2.7.2 tag and 
package for integration. 
[Here|https://github.com/datastax/python-driver/compare/2.7.2-cassandra-2.1] is 
a branch with your patch applied 
([refresher|https://github.com/datastax/python-driver/blob/master/README-dev.rst#packaging-for-cassandra]
 on how to roll with commit included). It looks like this change breaks [one of 
our *contrived* legacy metadata 
tests|https://github.com/datastax/python-driver/blob/2.7.2-cassandra-2.1/tests/integration/standard/test_metadata.py#L869-L1133],
 so we'll have to decide if the test of the change is wrong. I'm out of office 
W-F this week so I'm not sure I'll find the time to work on it.

I will also take a look to see if this schema impacts legacy schema parsing in 
recent versions of the driver connecting to older versions of Cassandra. That 
should not be a blocker for this ticket though. Follow-on for the driver: 
[PYTHON-562|https://datastax-oss.atlassian.net/browse/PYTHON-562]

> C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no 
> attribute 'replace'
> ---
>
> Key: CASSANDRA-11055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11055
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Simon Ashley
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: 11055-driver-2.7.2.patch, data.tar.gz
>
>
> C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns:
>  'NoneType' object has no attribute 'replace' 
> for thrift CF's originally created in C* 1.2.
> Repro:
> 1. Create cf in cassandra-cli on C* 1.2.x  (1.2.9 was used here)
> [default@ks1] CREATE COLUMN FAMILY t1
> ...   WITH column_type='Standard'
> ...   AND 
> comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> ...   AND default_validation_class='UTF8Type'
> ...   AND key_validation_class='UTF8Type'
> ...   AND read_repair_chance=0.1
> ...   AND dclocal_read_repair_chance=0.0
> ...   AND gc_grace=864000
> ...   AND min_compaction_threshold=4
> ...   AND max_compaction_threshold=32
> ...   AND replicate_on_write=true
> ...   AND compaction_strategy='LeveledCompactionStrategy' AND 
> compaction_strategy_options={sstable_size_in_mb: 32}
> ...   AND caching='KEYS_ONLY'
> ...   AND compression_options={sstable_compression:SnappyCompressor, 
> chunk_length_kb:64};
> qlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {
>   'class': 'NetworkTopologyStrategy',
>   'datacenter1': '1'
> };
> USE ks1;
> CREATE TABLE t1 (
>   key text,
>   column1 text,
>   column2 text,
>   value text,
>   PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE AND
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=864000 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '32', 'class': 
> 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '64', 'sstable_compression': 
> 'SnappyCompressor'};
> cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases 
> from system.schema_columnfamilies where keyspace_name= 'ks1';
>  keyspace_name | columnfamily_name | column_aliases | key_aliases
> ---+---++-
>ks1 |t1 | [] |  []
> 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a
> At this stage , DESCRIBE in cqlsh is working
> 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a
> DESCRIBE now fails:
> cqlsh> describe table ks1.t1;
> 'NoneType' object has no attribute 'replace'
> cqlsh> describe keyspace ks1;
> 'NoneType' object has no attribute 'replace'
> You can workaround by manually updating system.schema_columnfamilies
>  UPDATE system.schema_columnfamilies SET column_aliases 
> ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = 
> 't1';
> Once you exit and restart cqlsh, DESCRIBE is not working as per C* 1.2
> cqlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1'}  AND durable_writes = true;
> CREATE TABLE ks1.t1 (
> key text,
> column1 text,

[jira] [Updated] (CASSANDRA-9530) SSTable corruption can trigger OOM

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9530:
---
Reviewer: Stefania

> SSTable corruption can trigger OOM
> --
>
> Key: CASSANDRA-9530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9530
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
>
> If a sstable is corrupted so that the length of a given is bogus, we'll still 
> happily try to allocate a buffer of that bogus size to read the value, which 
> can easily lead to an OOM.
> We should probably protect against this. In practice, a given value can be so 
> big since it's limited by the protocol frame size in the first place. Maybe 
> we could add a max_value_size_in_mb setting and we'd considered a sstable 
> corrupted if it was containing a value bigger than that.
> I'll note that this ticket would be a good occasion to improve 
> {{BlacklistingCompactionsTest}}. Typically, it currently generate empty 
> values which makes it pretty much impossible to get the problem described 
> here. And as described in CASSANDRA-9478, it also doesn't test properly for 
> thing like early opening of compaction results. We could try to randomize as 
> much of the parameters of this test as possible to make it more likely to 
> catch any type of corruption that could happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11580) remove DatabaseDescriptor dependency from SegmentedFile

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11580:

Reviewer: Stefania

> remove DatabaseDescriptor dependency from SegmentedFile
> ---
>
> Key: CASSANDRA-11580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11580
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
> Fix For: 3.x
>
>
> Several configurable parameters are pulled from {{DatabaseDescriptor}} from 
> {{SegmentedFile}} and its subclasses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11670) Error while waiting on bootstrap to complete. Bootstrap will have to be restarted. Stream failed

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11670:

Status: Patch Available  (was: In Progress)

> Error while waiting on bootstrap to complete. Bootstrap will have to be 
> restarted. Stream failed
> 
>
> Key: CASSANDRA-11670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11670
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration, Streaming and Messaging
>Reporter: Anastasia Osintseva
>Assignee: Paulo Motta
> Fix For: 3.0.5
>
>
> I have in cluster 2 DC, in each DC - 2 Nodes. I wanted to add 1 node to each 
> DC. One node has been added successfully after I had made scrubing. 
> Now I'm trying to add node to another DC, but get error: 
> org.apache.cassandra.streaming.StreamException: Stream failed. 
> After scrubing and repair I get the same error.  
> {noformat}
> ERROR [StreamReceiveTask:5] 2016-04-27 00:33:21,082 Keyspace.java:492 - 
> Unknown exception caught while attempting to update MaterializedView! 
> messages_dump.messages
> java.lang.IllegalArgumentException: Mutation of 34974901 bytes is too large 
> for the maxiumum size of 33554432
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:264) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:469) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:205) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:217) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.batchlog.BatchlogManager.store(BatchlogManager.java:146) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:724) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:149)
>  ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:487) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:205) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:217) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:236) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:169)
>  [apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_11]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_11]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_11]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_11]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
> ERROR [StreamReceiveTask:5] 2016-04-27 00:33:21,082 
> StreamReceiveTask.java:214 - Error applying streamed data: 
> java.lang.IllegalArgumentException: Mutation of 34974901 bytes is too large 
> for the maxiumum size of 33554432
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:264) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:469) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:205) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:217) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.batchlog.BatchlogManager.store(BatchlogManager.java:146) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:724) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:149)
>  ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:487) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at 

[jira] [Commented] (CASSANDRA-11670) Error while waiting on bootstrap to complete. Bootstrap will have to be restarted. Stream failed

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290085#comment-15290085
 ] 

Paulo Motta commented on CASSANDRA-11670:
-

Attaching initial patch for review.

Basic idea is to introduce a {{BatchlogBuilder}}, that can be instantiated with 
a maximum size, and creates new batchlogs as mutation are added to it if it 
exceeds the specified maximum size. Based on this, I added a 
{{BatchlogManager.storeMultiBatch}} method that receives a list of mutations 
and stores them in multiple batches, respecting the maximum size of 
(90%*max_mutation_size_in_kb) per batch. This method is used when [storing 
views on a bootstrapping or moving 
node|https://github.com/pauloricardomg/cassandra/blob/63c5001e7cde2a6296fe3b14e96bb9225d893585/src/java/org/apache/cassandra/service/StorageProxy.java#L721],
 since we skip the normal MV path if there are pending endpoint.

When adding mutations via the normal MV path, a {{BatchlogBuilder}} is created 
and [mutations are appended to 
it|https://github.com/pauloricardomg/cassandra/blob/63c5001e7cde2a6296fe3b14e96bb9225d893585/src/java/org/apache/cassandra/service/StorageProxy.java#L738]
 as the view write response handlers are created. Each time a new batchlog is 
created, a new {{BatchlogCleanup}} callback object is 
[created|https://github.com/pauloricardomg/cassandra/blob/63c5001e7cde2a6296fe3b14e96bb9225d893585/src/java/org/apache/cassandra/service/StorageProxy.java#L742],
 so it removes each batchlog independently after all of its mutations are 
written. Since we don't know the amount of mutations a batchlog will contain 
ahead of time, we increase the mutation count in the {{BatchlogCleanup}} as 
they are added to the {{BatchlogBuilder}}.

I noticed that when we apply the view mutation locally we don't currently 
decrease the {{mutationsWaitingFor}} on {{BatchlogCleanup}}, so does this mean 
we never clean up view batchlogs when there are local paired mutations? In 
order to avoid this in the new approach I incremented the number of mutations 
in the {{BatchlogCleanup}} [only if the replica is not applied 
locally|https://github.com/pauloricardomg/cassandra/blob/63c5001e7cde2a6296fe3b14e96bb9225d893585/src/java/org/apache/cassandra/service/StorageProxy.java#L766].
 Can we further optimize this and only add the replica to the batchlog if it's 
not local?

I added a 
[dtest|https://github.com/riptano/cassandra-dtest/compare/master...pauloricardomg:11670]
 that reproduces this issue and it is fixed by this approach.  If it looks good 
I will add more thorough testing and documentation to {{BatchlogBuilder}}.

Patch and tests available below:

||3.0||dtest||
|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-11670]|[branch|https://github.com/riptano/cassandra-dtest/compare/master...pauloricardomg:11670]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-11670-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-11670-dtest/lastCompletedBuild/testReport/]|

> Error while waiting on bootstrap to complete. Bootstrap will have to be 
> restarted. Stream failed
> 
>
> Key: CASSANDRA-11670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11670
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration, Streaming and Messaging
>Reporter: Anastasia Osintseva
>Assignee: Paulo Motta
> Fix For: 3.0.5
>
>
> I have in cluster 2 DC, in each DC - 2 Nodes. I wanted to add 1 node to each 
> DC. One node has been added successfully after I had made scrubing. 
> Now I'm trying to add node to another DC, but get error: 
> org.apache.cassandra.streaming.StreamException: Stream failed. 
> After scrubing and repair I get the same error.  
> {noformat}
> ERROR [StreamReceiveTask:5] 2016-04-27 00:33:21,082 Keyspace.java:492 - 
> Unknown exception caught while attempting to update MaterializedView! 
> messages_dump.messages
> java.lang.IllegalArgumentException: Mutation of 34974901 bytes is too large 
> for the maxiumum size of 33554432
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:264) 
> ~[apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:469) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:205) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:217) 
> [apache-cassandra-3.0.5.jar:3.0.5]
>   at 
> 

[jira] [Updated] (CASSANDRA-11842) Unbounded commit log file growth

[Andrew Jorgensen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Jorgensen updated CASSANDRA-11842:
-
Description: 
Today I noticed that 2 nodes in a 54 node cluster have been using up disk space 
at a constant rate for the last 3 days or so. 

This is a graph of disk space over the last 4 days for each of the nodes in our 
cassandra cluster.
!disks-space.png|thumnnail!

When I looked into it I found that the majority of the disk space was being 
used up in /mnt/cassandra/commitlog. It looked like there were files dating 
back to when the disk usage started to increase on 5/16 and there were a total 
of ~13K commit log files in this directory.

I was curious if anyone has seen this before. I am not sure what would cause 
this behavior, especially on two separate nodes in the cluster at about the 
same time. I think this points to something about the data, we have a 
replication factor of 2 which seems to match up with the number of nodes that 
were affected.

The two nodes in question looked down from every other node in the clusters 
perspective when doing `nodetool` status but when running that on the affected 
nodes the entire cluster looked  like it was up and running.

To remedy the situation I tried running `nodetool drain` on one of the affected 
nodes but it seemed to be hung and I couldnt get a handle on if it was doing 
anything or not. I restarted the cassandra process and could see in the debug 
log that it was reading in the commit log files. After it finished reading all 
of the commit log files it correctly removed the files and restored the disk 
space. On the second node I moved the commit log folder to a different location 
and restarted the node which cause it to immediately rejoin the cluster and I 
can go re-play the commit log files that were queued up later to make sure its 
in a consistent state. So far it looks like the commit log file growth on that 
node is not growing unboundedly.

As far as I could tell the data in /mnt/cassandra/data/ for each of the 
keyspaces and tables had recent timestamps on the file which I believe means 
that flushing was happening and data was getting written to the SStables, also 
350GB of commitlog wouldnt have been able to fit into memory.

In terms of settings we do not have `commitlog_total_space_in_mb` set so it 
should be whatever the default is. We do have the 
`commitlog_segment_size_in_mb` set to 32.

If there is any other information I can provide please let me know. I didnt see 
much in the cassandra system.log or debug.log file but would be happy to 
provide them if it'll help.

  was:
Today I noticed that 2 nodes in a 54 node cluster have been using up disk space 
at a constant rate for the last 3 days or so. 

This is a graph of disk space over the last 4 days for each of the nodes in our 
cassandra cluster.
!disks-space.png|thumnnail!

When I looked into it I found that the majority of the disk space was being 
used up in /mnt/cassandra/commitlog. It looked like there were files dating 
back to when the disk usage started to increase on 5/16 and there were a total 
of ~13K commit log files in this directory.

I was curious if anyone has seen this before. I am not sure what would cause 
this behavior, especially on two separate nodes in the cluster at about the 
same time. I think this points to something about the data, we have a 
replication factor of 2 which seems to match up with the number of nodes that 
were affected.

The two nodes in question looked down from every other node in the clusters 
perspective when doing `nodetool` status but when running that on the affected 
nodes the entire cluster looked  like it was up and running.

To remedy the situation I tried running `nodetool drain` on one of the affected 
nodes but it seemed to be hung and I couldnt get a handle on if it was doing 
anything or not. I restarted the cassandra process and could see in the debug 
log that it was reading in the commit log files. On the second node I moved the 
commit log folder to a different location and restarted the node which cause it 
to immediately rejoin the cluster and I can go re-play the commit log files 
that were queued up later to make sure its in a consistent state. So far it 
looks like the commit log file growth on that node is not growing unboundedly.

As far as I could tell the data in /mnt/cassandra/data/ for each of the 
keyspaces and tables had recent timestamps on the file which I believe means 
that flushing was happening and data was getting written to the SStables, also 
350GB of commitlog wouldnt have been able to fit into memory.

If there is any other information I can provide please let me know. I didnt see 
much in the cassandra system.log or debug.log file but would be happy to 
provide them if it'll help.


> Unbounded commit log file growth
> 
>
> 

[jira] [Created] (CASSANDRA-11842) Unbounded commit log file growth

[Andrew Jorgensen (JIRA)]

Andrew Jorgensen created CASSANDRA-11842:


 Summary: Unbounded commit log file growth
 Key: CASSANDRA-11842
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11842
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 3.0.3 on Ubuntu Trusty
Reporter: Andrew Jorgensen
 Attachments: disks-space.png

Today I noticed that 2 nodes in a 54 node cluster have been using up disk space 
at a constant rate for the last 3 days or so. 

!disks-space.png|thumnnail!

When I looked into it I found that the majority of the disk space was being 
used up in /mnt/cassandra/commitlog. It looked like there were files dating 
back to when the disk usage started to increase on 5/16 and there were a total 
of ~13K commit log files in this directory.

I was curious if anyone has seen this before. I am not sure what would cause 
this behavior, especially on two separate nodes in the cluster at about the 
same time. I think this points to something about the data, we have a 
replication factor of 2 which seems to match up with the number of nodes that 
were affected.

The two nodes in question looked down from every other node in the clusters 
perspective when doing `nodetool` status but when running that on the affected 
nodes the entire cluster looked  like it was up and running.

To remedy the situation I tried running `nodetool drain` on one of the affected 
nodes but it seemed to be hung and I couldnt get a handle on if it was doing 
anything or not. I restarted the cassandra process and could see in the debug 
log that it was reading in the commit log files. On the second node I moved the 
commit log folder to a different location and restarted the node which cause it 
to immediately rejoin the cluster and I can go re-play the commit log files 
that were queued up later to make sure its in a consistent state. So far it 
looks like the commit log file growth on that node is not growing unboundedly.

As far as I could tell the data in /mnt/cassandra/data/ for each of the 
keyspaces and tables had recent timestamps on the file which I believe means 
that flushing was happening and data was getting written to the SStables, also 
350GB of commitlog wouldnt have been able to fit into memory.

If there is any other information I can provide please let me know. I didnt see 
much in the cassandra system.log or debug.log file but would be happy to 
provide them if it'll help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11842) Unbounded commit log file growth

[Andrew Jorgensen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Jorgensen updated CASSANDRA-11842:
-
Description: 
Today I noticed that 2 nodes in a 54 node cluster have been using up disk space 
at a constant rate for the last 3 days or so. 

This is a graph of disk space over the last 4 days for each of the nodes in our 
cassandra cluster.
!disks-space.png|thumnnail!

When I looked into it I found that the majority of the disk space was being 
used up in /mnt/cassandra/commitlog. It looked like there were files dating 
back to when the disk usage started to increase on 5/16 and there were a total 
of ~13K commit log files in this directory.

I was curious if anyone has seen this before. I am not sure what would cause 
this behavior, especially on two separate nodes in the cluster at about the 
same time. I think this points to something about the data, we have a 
replication factor of 2 which seems to match up with the number of nodes that 
were affected.

The two nodes in question looked down from every other node in the clusters 
perspective when doing `nodetool` status but when running that on the affected 
nodes the entire cluster looked  like it was up and running.

To remedy the situation I tried running `nodetool drain` on one of the affected 
nodes but it seemed to be hung and I couldnt get a handle on if it was doing 
anything or not. I restarted the cassandra process and could see in the debug 
log that it was reading in the commit log files. On the second node I moved the 
commit log folder to a different location and restarted the node which cause it 
to immediately rejoin the cluster and I can go re-play the commit log files 
that were queued up later to make sure its in a consistent state. So far it 
looks like the commit log file growth on that node is not growing unboundedly.

As far as I could tell the data in /mnt/cassandra/data/ for each of the 
keyspaces and tables had recent timestamps on the file which I believe means 
that flushing was happening and data was getting written to the SStables, also 
350GB of commitlog wouldnt have been able to fit into memory.

If there is any other information I can provide please let me know. I didnt see 
much in the cassandra system.log or debug.log file but would be happy to 
provide them if it'll help.

  was:
Today I noticed that 2 nodes in a 54 node cluster have been using up disk space 
at a constant rate for the last 3 days or so. 

!disks-space.png|thumnnail!

When I looked into it I found that the majority of the disk space was being 
used up in /mnt/cassandra/commitlog. It looked like there were files dating 
back to when the disk usage started to increase on 5/16 and there were a total 
of ~13K commit log files in this directory.

I was curious if anyone has seen this before. I am not sure what would cause 
this behavior, especially on two separate nodes in the cluster at about the 
same time. I think this points to something about the data, we have a 
replication factor of 2 which seems to match up with the number of nodes that 
were affected.

The two nodes in question looked down from every other node in the clusters 
perspective when doing `nodetool` status but when running that on the affected 
nodes the entire cluster looked  like it was up and running.

To remedy the situation I tried running `nodetool drain` on one of the affected 
nodes but it seemed to be hung and I couldnt get a handle on if it was doing 
anything or not. I restarted the cassandra process and could see in the debug 
log that it was reading in the commit log files. On the second node I moved the 
commit log folder to a different location and restarted the node which cause it 
to immediately rejoin the cluster and I can go re-play the commit log files 
that were queued up later to make sure its in a consistent state. So far it 
looks like the commit log file growth on that node is not growing unboundedly.

As far as I could tell the data in /mnt/cassandra/data/ for each of the 
keyspaces and tables had recent timestamps on the file which I believe means 
that flushing was happening and data was getting written to the SStables, also 
350GB of commitlog wouldnt have been able to fit into memory.

If there is any other information I can provide please let me know. I didnt see 
much in the cassandra system.log or debug.log file but would be happy to 
provide them if it'll help.


> Unbounded commit log file growth
> 
>
> Key: CASSANDRA-11842
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11842
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra version 3.0.3 on Ubuntu Trusty
>Reporter: Andrew Jorgensen
> Attachments: disks-space.png
>
>
> Today I noticed that 2 nodes in a 54 node cluster have been using up disk 
> 

[Cassandra Wiki] Update of "GettingStarted" by TylerHobbs

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "GettingStarted" page has been changed by TylerHobbs:
https://wiki.apache.org/cassandra/GettingStarted?action=diff=107=108

Comment:
Specify that Java 8 is required for Cassandra 3.0+

  This document aims to provide a few easy to follow steps to take the 
first-time user from installation, to running single node Cassandra, and 
overview to configure multinode cluster. Cassandra is meant to run on a cluster 
of nodes, but will run equally well on a single machine. This is a handy way of 
getting familiar with the software while avoiding the complexities of a larger 
system.
  
  == Step 0: Prerequisites and Connecting to the Community ==
- Cassandra requires the most stable version of Java 7 or 8 you can deploy, 
preferably the 
[[http://www.oracle.com/technetwork/java/javase/downloads/index.html|Oracle/Sun 
JVM]].  Cassandra also runs on OpenJDK, Zing, and the IBM JVM.  (It will NOT 
run on JRockit, which is only compatible with Java 6.)
+ Cassandra 3.0+ requires the most stable version of Java 8 you can deploy, 
preferably the 
[[http://www.oracle.com/technetwork/java/javase/downloads/index.html|Oracle/Sun 
JVM]].  Cassandra also runs on OpenJDK, Zing, and the IBM JVM.  (It will NOT 
run on JRockit, which is only compatible with Java 6.)
  
  The best way to ensure you always have up to date information on the project, 
releases, stability, bugs, and features is to subscribe to the users mailing 
list ([[mailto:user-subscr...@cassandra.apache.org|subscription required]]) and 
participate in the #cassandra channel on 
[[http://webchat.freenode.net/?channels=#cassandra|IRC]].
  

[jira] [Commented] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289824#comment-15289824
 ] 

Benedict commented on CASSANDRA-9669:
-

Committed as 2837ec65b91abd78ec1bb37f1d69565b976e42e6

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-9669:

   Resolution: Fixed
Reproduced In: 2.1.7, 2.0.16, 1.2.19  (was: 1.2.19, 2.0.16, 2.1.7)
   Status: Resolved  (was: Ready to Commit)

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11569) Track message latency across DCs

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289810#comment-15289810
 ] 

Chris Lohfink commented on CASSANDRA-11569:
---

bq. I will got an error as the crosslatency is too large

What error?

I could not reproduce anything. I added some tests to make it easier to try 
different latencies as well (v2 patch with minor refactoring for easier 
testing).

> Track message latency across DCs
> 
>
> Key: CASSANDRA-11569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11569
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11569.patch, CASSANDRA-11569v2.txt
>
>
> Since we have the timestamp a message is created and when arrives, we can get 
> an approximate time it took relatively easy and would remove necessity for 
> more complex hacks to determine latency between DCs.
> Although is not going to be very meaningful when ntp is not setup, it is 
> pretty common to have NTP setup and even with clock drift nothing is really 
> hurt except the metric becoming whacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11569) Track message latency across DCs

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-11569:
--
Attachment: CASSANDRA-11569v2.txt

> Track message latency across DCs
> 
>
> Key: CASSANDRA-11569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11569
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Attachments: CASSANDRA-11569.patch, CASSANDRA-11569v2.txt
>
>
> Since we have the timestamp a message is created and when arrives, we can get 
> an approximate time it took relatively easy and would remove necessity for 
> more complex hacks to determine latency between DCs.
> Although is not going to be very meaningful when ntp is not setup, it is 
> pretty common to have NTP setup and even with clock drift nothing is really 
> hurt except the metric becoming whacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[6/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7

[benedict]

Merge branch 'cassandra-3.0' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/326a263f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/326a263f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/326a263f

Branch: refs/heads/cassandra-3.7
Commit: 326a263f4b1cc576226d89bcfbb561385ba54e10
Parents: 424b0b9 5a5d0a1
Author: Benedict Elliott Smith 
Authored: Wed May 18 22:00:16 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 22:00:16 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 41 +---
 1 file changed, 19 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/326a263f/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--

[3/8] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[benedict]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/958183fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/958183fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/958183fa

Branch: refs/heads/cassandra-3.7
Commit: 958183fadab6a67abf8831fa8e92751039a2ef58
Parents: f73b485 be65393
Author: Jonathan Ellis 
Authored: Wed May 18 12:30:54 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:54 2016 -0500

--

--

[8/8] cassandra git commit: Merge branch 'cassandra-3.7' into trunk

[benedict]

Merge branch 'cassandra-3.7' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/beb6464c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/beb6464c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/beb6464c

Branch: refs/heads/trunk
Commit: beb6464c0523155b03cd426f32fae8020d72df0f
Parents: bc78466 326a263
Author: Benedict Elliott Smith 
Authored: Wed May 18 22:00:27 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 22:00:27 2016 +0100

--

--

[5/8] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[benedict]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a5d0a1e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a5d0a1e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a5d0a1e

Branch: refs/heads/cassandra-3.7
Commit: 5a5d0a1eb46dfa308acc9cff28f8c65fbb026b81
Parents: 958183f 2837ec6
Author: Benedict Elliott Smith 
Authored: Wed May 18 21:51:40 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:51:40 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 41 +---
 1 file changed, 19 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a5d0a1e/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 98ba781,45486c1..a6d5c17
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1973,35 -2762,30 +1973,32 @@@ public class ColumnFamilyStore implemen
  final long truncatedAt;
  final ReplayPosition replayAfter;
  
- synchronized (data)
 -if (keyspace.getMetadata().durableWrites || takeSnapshot)
++if (keyspace.getMetadata().params.durableWrites || 
DatabaseDescriptor.isAutoSnapshot())
+ {
+ replayAfter = forceBlockingFlush();
++viewManager.forceBlockingFlush();
+ }
+ else
  {
- if (keyspace.getMetadata().params.durableWrites || 
DatabaseDescriptor.isAutoSnapshot())
+ // just nuke the memtable data w/o writing to disk first
 -Future replayAfterFuture;
 -synchronized (data)
++viewManager.dumpMemtables();
++try
  {
- replayAfter = forceBlockingFlush();
- viewManager.forceBlockingFlush();
 -final Flush flush = new Flush(true);
 -flushExecutor.execute(flush);
 -replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
++replayAfter = dumpMemtable().get();
 +}
- else
++catch (Exception e)
 +{
- // just nuke the memtable data w/o writing to disk first
- viewManager.dumpMemtables();
- try
- {
- replayAfter = dumpMemtable().get();
- }
- catch (Exception e)
- {
- throw new RuntimeException(e);
- }
++throw new RuntimeException(e);
  }
- 
- long now = System.currentTimeMillis();
- // make sure none of our sstables are somehow in the future 
(clock drift, perhaps)
- for (ColumnFamilyStore cfs : concatWithIndexes())
- for (SSTableReader sstable : cfs.data.getSSTables())
- now = Math.max(now, sstable.maxDataAge);
- truncatedAt = now;
 -replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
  }
  
+ long now = System.currentTimeMillis();
+ // make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+ for (ColumnFamilyStore cfs : concatWithIndexes())
+ for (SSTableReader sstable : cfs.data.getSSTables())
+ now = Math.max(now, sstable.maxDataAge);
+ truncatedAt = now;
+ 
  Runnable truncateRunnable = new Runnable()
  {
  public void run()

[2/8] cassandra git commit: merge from 2.1

[benedict]

merge from 2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be653932
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be653932
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be653932

Branch: refs/heads/cassandra-3.7
Commit: be653932d883134ec6ebecd5b9ad1dbb64bda64a
Parents: 89344f1 00f2540
Author: Jonathan Ellis 
Authored: Wed May 18 12:28:45 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:14 2016 -0500

--
 CHANGES.txt | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/be653932/CHANGES.txt
--
diff --cc CHANGES.txt
index befb520,3972248..103d057
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,8 +1,24 @@@
-  * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
- 
 -2.1.15
 +2.2.7
++ * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+expired tombstone (CASSANDRA-11834)
 - * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
 -   (CASSANDRA-11679)
   * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
   * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
   * Make cython optional in pylib/setup.py (CASSANDRA-11630)
@@@ -25,62 -11,21 +27,63 @@@
   * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
   * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
   * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
 + * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 +   processes (CASSANDRA-11505)
-  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+sstables (CASSANDRA-9935)
  
  
 -2.1.14
 - * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 -   going (CASSANDRA-10979)
 - * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 -   processes (CASSANDRA-11505)
 - * Replace sstables on DataTracker before marking them as non-compacting 
during anti-compaction (CASSANDRA-11548)
 +2.2.6
 + * Allow only DISTINCT queries with partition keys restrictions 
(CASSANDRA-11339)
 + * CqlConfigHelper no longer requires both a keystore and truststore to work 
(CASSANDRA-11532)
 + * Make deprecated repair methods backward-compatible with previous 
notification service (CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
 + * DatabaseDescriptor should log stacktrace in case of Eception during seed 
provider creation (CASSANDRA-11312)
 + * Use canonical path for directory in SSTable descriptor (CASSANDRA-10587)
 + * Add cassandra-stress keystore option (CASSANDRA-9325)
 + * Fix out-of-space error treatment in memtable flushing (CASSANDRA-11448).
 + * Dont mark sstables as repairing with sub range repairs (CASSANDRA-11451)
 + * Fix use of NullUpdater for 2i during compaction (CASSANDRA-11450)
 + * Notify when sstables change after cancelling compaction (CASSANDRA-11373)
 + * cqlsh: COPY FROM should check that explicit column names are valid 
(CASSANDRA-11333)
 + * Add -Dcassandra.start_gossip 

[7/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7

[benedict]

Merge branch 'cassandra-3.0' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/326a263f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/326a263f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/326a263f

Branch: refs/heads/trunk
Commit: 326a263f4b1cc576226d89bcfbb561385ba54e10
Parents: 424b0b9 5a5d0a1
Author: Benedict Elliott Smith 
Authored: Wed May 18 22:00:16 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 22:00:16 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 41 +---
 1 file changed, 19 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/326a263f/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--

[4/8] cassandra git commit: Fix deadlock on truncation with secondary index.

[benedict]

Fix deadlock on truncation with secondary index.

Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2837ec65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2837ec65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2837ec65

Branch: refs/heads/cassandra-3.7
Commit: 2837ec65b91abd78ec1bb37f1d69565b976e42e6
Parents: be65393
Author: Branimir Lambov 
Authored: Tue May 17 12:22:21 2016 +0300
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:49:39 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 32 +++-
 1 file changed, 17 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2837ec65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 88e22c0..45486c1 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2762,28 +2762,30 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 final long truncatedAt;
 final ReplayPosition replayAfter;
 
-synchronized (data)
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
 {
-if (keyspace.getMetadata().durableWrites || takeSnapshot)
-{
-replayAfter = forceBlockingFlush();
-}
-else
+replayAfter = forceBlockingFlush();
+}
+else
+{
+// just nuke the memtable data w/o writing to disk first
+Future replayAfterFuture;
+synchronized (data)
 {
-// just nuke the memtable data w/o writing to disk first
 final Flush flush = new Flush(true);
 flushExecutor.execute(flush);
-replayAfter = 
FBUtilities.waitOnFuture(postFlushExecutor.submit(flush.postFlush));
+replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
 }
-
-long now = System.currentTimeMillis();
-// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
-for (ColumnFamilyStore cfs : concatWithIndexes())
-for (SSTableReader sstable : cfs.data.getSSTables())
-now = Math.max(now, sstable.maxDataAge);
-truncatedAt = now;
+replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
 }
 
+long now = System.currentTimeMillis();
+// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+for (ColumnFamilyStore cfs : concatWithIndexes())
+for (SSTableReader sstable : cfs.data.getSSTables())
+now = Math.max(now, sstable.maxDataAge);
+truncatedAt = now;
+
 Runnable truncateRunnable = new Runnable()
 {
 public void run()

[1/8] cassandra git commit: Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone patch by jbellis; reviewed by marcuse for CASSANDRA-11834

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.7 424b0b9e0 -> 326a263f4
  refs/heads/trunk bc7846680 -> beb6464c0


Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
expired tombstone
patch by jbellis; reviewed by marcuse for CASSANDRA-11834


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00f25401
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00f25401
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00f25401

Branch: refs/heads/cassandra-3.7
Commit: 00f25401c9a320c0c2ac94fbb1014f49597026fd
Parents: 9359af2
Author: Jonathan Ellis 
Authored: Wed May 18 12:24:23 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:25:45 2016 -0500

--
 CHANGES.txt  | 11 ---
 .../cassandra/db/compaction/LazilyCompactedRow.java  |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 68f8dd9..3972248 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,13 +1,18 @@
 2.1.15
- * Fix paging on DISTINCT queries repeats result when first row in partition 
changes (CASSANDRA-11679)
+ * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+   expired tombstone (CASSANDRA-11834)
+ * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
+   (CASSANDRA-11679)
  * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
  * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
  * Make cython optional in pylib/setup.py (CASSANDRA-11630)
- * Change order of directory searching for cassandra.in.sh to favor local one 
(CASSANDRA-11628)
+ * Change order of directory searching for cassandra.in.sh to favor local one 
+   (CASSANDRA-11628)
  * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
  * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
  * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
- * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+ * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+   sstables (CASSANDRA-9935)
 
 
 2.1.14

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java 
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index 941557b..4ca9829 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -271,7 +271,7 @@ public class LazilyCompactedRow extends AbstractCompactedRow
 RangeTombstone t = tombstone;
 tombstone = null;
 
-if (t.timestamp() < getMaxPurgeableTimestamp() && 
t.data.isGcAble(controller.gcBefore))
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() < 
getMaxPurgeableTimestamp())
 {
 indexBuilder.tombstoneTracker().update(t, true);
 return null;

[6/6] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[benedict]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bc784668
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bc784668
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bc784668

Branch: refs/heads/trunk
Commit: bc7846680e82b2c728adbc1f35e11c09c659dd33
Parents: 5bdf318 5a5d0a1
Author: Benedict Elliott Smith 
Authored: Wed May 18 21:51:57 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:51:57 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 41 +---
 1 file changed, 19 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bc784668/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--

[3/6] cassandra git commit: Fix deadlock on truncation with secondary index.

[benedict]

Fix deadlock on truncation with secondary index.

Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2837ec65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2837ec65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2837ec65

Branch: refs/heads/trunk
Commit: 2837ec65b91abd78ec1bb37f1d69565b976e42e6
Parents: be65393
Author: Branimir Lambov 
Authored: Tue May 17 12:22:21 2016 +0300
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:49:39 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 32 +++-
 1 file changed, 17 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2837ec65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 88e22c0..45486c1 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2762,28 +2762,30 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 final long truncatedAt;
 final ReplayPosition replayAfter;
 
-synchronized (data)
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
 {
-if (keyspace.getMetadata().durableWrites || takeSnapshot)
-{
-replayAfter = forceBlockingFlush();
-}
-else
+replayAfter = forceBlockingFlush();
+}
+else
+{
+// just nuke the memtable data w/o writing to disk first
+Future replayAfterFuture;
+synchronized (data)
 {
-// just nuke the memtable data w/o writing to disk first
 final Flush flush = new Flush(true);
 flushExecutor.execute(flush);
-replayAfter = 
FBUtilities.waitOnFuture(postFlushExecutor.submit(flush.postFlush));
+replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
 }
-
-long now = System.currentTimeMillis();
-// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
-for (ColumnFamilyStore cfs : concatWithIndexes())
-for (SSTableReader sstable : cfs.data.getSSTables())
-now = Math.max(now, sstable.maxDataAge);
-truncatedAt = now;
+replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
 }
 
+long now = System.currentTimeMillis();
+// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+for (ColumnFamilyStore cfs : concatWithIndexes())
+for (SSTableReader sstable : cfs.data.getSSTables())
+now = Math.max(now, sstable.maxDataAge);
+truncatedAt = now;
+
 Runnable truncateRunnable = new Runnable()
 {
 public void run()

[4/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[benedict]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a5d0a1e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a5d0a1e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a5d0a1e

Branch: refs/heads/trunk
Commit: 5a5d0a1eb46dfa308acc9cff28f8c65fbb026b81
Parents: 958183f 2837ec6
Author: Benedict Elliott Smith 
Authored: Wed May 18 21:51:40 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:51:40 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 41 +---
 1 file changed, 19 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a5d0a1e/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 98ba781,45486c1..a6d5c17
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1973,35 -2762,30 +1973,32 @@@ public class ColumnFamilyStore implemen
  final long truncatedAt;
  final ReplayPosition replayAfter;
  
- synchronized (data)
 -if (keyspace.getMetadata().durableWrites || takeSnapshot)
++if (keyspace.getMetadata().params.durableWrites || 
DatabaseDescriptor.isAutoSnapshot())
+ {
+ replayAfter = forceBlockingFlush();
++viewManager.forceBlockingFlush();
+ }
+ else
  {
- if (keyspace.getMetadata().params.durableWrites || 
DatabaseDescriptor.isAutoSnapshot())
+ // just nuke the memtable data w/o writing to disk first
 -Future replayAfterFuture;
 -synchronized (data)
++viewManager.dumpMemtables();
++try
  {
- replayAfter = forceBlockingFlush();
- viewManager.forceBlockingFlush();
 -final Flush flush = new Flush(true);
 -flushExecutor.execute(flush);
 -replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
++replayAfter = dumpMemtable().get();
 +}
- else
++catch (Exception e)
 +{
- // just nuke the memtable data w/o writing to disk first
- viewManager.dumpMemtables();
- try
- {
- replayAfter = dumpMemtable().get();
- }
- catch (Exception e)
- {
- throw new RuntimeException(e);
- }
++throw new RuntimeException(e);
  }
- 
- long now = System.currentTimeMillis();
- // make sure none of our sstables are somehow in the future 
(clock drift, perhaps)
- for (ColumnFamilyStore cfs : concatWithIndexes())
- for (SSTableReader sstable : cfs.data.getSSTables())
- now = Math.max(now, sstable.maxDataAge);
- truncatedAt = now;
 -replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
  }
  
+ long now = System.currentTimeMillis();
+ // make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+ for (ColumnFamilyStore cfs : concatWithIndexes())
+ for (SSTableReader sstable : cfs.data.getSSTables())
+ now = Math.max(now, sstable.maxDataAge);
+ truncatedAt = now;
+ 
  Runnable truncateRunnable = new Runnable()
  {
  public void run()

[2/6] cassandra git commit: Fix deadlock on truncation with secondary index.

[benedict]

Fix deadlock on truncation with secondary index.

Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2837ec65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2837ec65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2837ec65

Branch: refs/heads/cassandra-3.0
Commit: 2837ec65b91abd78ec1bb37f1d69565b976e42e6
Parents: be65393
Author: Branimir Lambov 
Authored: Tue May 17 12:22:21 2016 +0300
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:49:39 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 32 +++-
 1 file changed, 17 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2837ec65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 88e22c0..45486c1 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2762,28 +2762,30 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 final long truncatedAt;
 final ReplayPosition replayAfter;
 
-synchronized (data)
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
 {
-if (keyspace.getMetadata().durableWrites || takeSnapshot)
-{
-replayAfter = forceBlockingFlush();
-}
-else
+replayAfter = forceBlockingFlush();
+}
+else
+{
+// just nuke the memtable data w/o writing to disk first
+Future replayAfterFuture;
+synchronized (data)
 {
-// just nuke the memtable data w/o writing to disk first
 final Flush flush = new Flush(true);
 flushExecutor.execute(flush);
-replayAfter = 
FBUtilities.waitOnFuture(postFlushExecutor.submit(flush.postFlush));
+replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
 }
-
-long now = System.currentTimeMillis();
-// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
-for (ColumnFamilyStore cfs : concatWithIndexes())
-for (SSTableReader sstable : cfs.data.getSSTables())
-now = Math.max(now, sstable.maxDataAge);
-truncatedAt = now;
+replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
 }
 
+long now = System.currentTimeMillis();
+// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+for (ColumnFamilyStore cfs : concatWithIndexes())
+for (SSTableReader sstable : cfs.data.getSSTables())
+now = Math.max(now, sstable.maxDataAge);
+truncatedAt = now;
+
 Runnable truncateRunnable = new Runnable()
 {
 public void run()

[1/6] cassandra git commit: Fix deadlock on truncation with secondary index.

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 be653932d -> 2837ec65b
  refs/heads/cassandra-3.0 958183fad -> 5a5d0a1eb
  refs/heads/trunk 5bdf3184b -> bc7846680


Fix deadlock on truncation with secondary index.

Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2837ec65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2837ec65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2837ec65

Branch: refs/heads/cassandra-2.2
Commit: 2837ec65b91abd78ec1bb37f1d69565b976e42e6
Parents: be65393
Author: Branimir Lambov 
Authored: Tue May 17 12:22:21 2016 +0300
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:49:39 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 32 +++-
 1 file changed, 17 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2837ec65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 88e22c0..45486c1 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2762,28 +2762,30 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 final long truncatedAt;
 final ReplayPosition replayAfter;
 
-synchronized (data)
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
 {
-if (keyspace.getMetadata().durableWrites || takeSnapshot)
-{
-replayAfter = forceBlockingFlush();
-}
-else
+replayAfter = forceBlockingFlush();
+}
+else
+{
+// just nuke the memtable data w/o writing to disk first
+Future replayAfterFuture;
+synchronized (data)
 {
-// just nuke the memtable data w/o writing to disk first
 final Flush flush = new Flush(true);
 flushExecutor.execute(flush);
-replayAfter = 
FBUtilities.waitOnFuture(postFlushExecutor.submit(flush.postFlush));
+replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
 }
-
-long now = System.currentTimeMillis();
-// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
-for (ColumnFamilyStore cfs : concatWithIndexes())
-for (SSTableReader sstable : cfs.data.getSSTables())
-now = Math.max(now, sstable.maxDataAge);
-truncatedAt = now;
+replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
 }
 
+long now = System.currentTimeMillis();
+// make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+for (ColumnFamilyStore cfs : concatWithIndexes())
+for (SSTableReader sstable : cfs.data.getSSTables())
+now = Math.max(now, sstable.maxDataAge);
+truncatedAt = now;
+
 Runnable truncateRunnable = new Runnable()
 {
 public void run()

[5/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[benedict]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a5d0a1e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a5d0a1e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a5d0a1e

Branch: refs/heads/cassandra-3.0
Commit: 5a5d0a1eb46dfa308acc9cff28f8c65fbb026b81
Parents: 958183f 2837ec6
Author: Benedict Elliott Smith 
Authored: Wed May 18 21:51:40 2016 +0100
Committer: Benedict Elliott Smith 
Committed: Wed May 18 21:51:40 2016 +0100

--
 .../apache/cassandra/db/ColumnFamilyStore.java  | 41 +---
 1 file changed, 19 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a5d0a1e/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 98ba781,45486c1..a6d5c17
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1973,35 -2762,30 +1973,32 @@@ public class ColumnFamilyStore implemen
  final long truncatedAt;
  final ReplayPosition replayAfter;
  
- synchronized (data)
 -if (keyspace.getMetadata().durableWrites || takeSnapshot)
++if (keyspace.getMetadata().params.durableWrites || 
DatabaseDescriptor.isAutoSnapshot())
+ {
+ replayAfter = forceBlockingFlush();
++viewManager.forceBlockingFlush();
+ }
+ else
  {
- if (keyspace.getMetadata().params.durableWrites || 
DatabaseDescriptor.isAutoSnapshot())
+ // just nuke the memtable data w/o writing to disk first
 -Future replayAfterFuture;
 -synchronized (data)
++viewManager.dumpMemtables();
++try
  {
- replayAfter = forceBlockingFlush();
- viewManager.forceBlockingFlush();
 -final Flush flush = new Flush(true);
 -flushExecutor.execute(flush);
 -replayAfterFuture = postFlushExecutor.submit(flush.postFlush);
++replayAfter = dumpMemtable().get();
 +}
- else
++catch (Exception e)
 +{
- // just nuke the memtable data w/o writing to disk first
- viewManager.dumpMemtables();
- try
- {
- replayAfter = dumpMemtable().get();
- }
- catch (Exception e)
- {
- throw new RuntimeException(e);
- }
++throw new RuntimeException(e);
  }
- 
- long now = System.currentTimeMillis();
- // make sure none of our sstables are somehow in the future 
(clock drift, perhaps)
- for (ColumnFamilyStore cfs : concatWithIndexes())
- for (SSTableReader sstable : cfs.data.getSSTables())
- now = Math.max(now, sstable.maxDataAge);
- truncatedAt = now;
 -replayAfter = FBUtilities.waitOnFuture(replayAfterFuture);
  }
  
+ long now = System.currentTimeMillis();
+ // make sure none of our sstables are somehow in the future (clock 
drift, perhaps)
+ for (ColumnFamilyStore cfs : concatWithIndexes())
+ for (SSTableReader sstable : cfs.data.getSSTables())
+ now = Math.max(now, sstable.maxDataAge);
+ truncatedAt = now;
+ 
  Runnable truncateRunnable = new Runnable()
  {
  public void run()

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289767#comment-15289767
 ] 

Robert Stupp commented on CASSANDRA-10786:
--

A v5 is fine IMO. But I'd suggest to include the v4-hash plus a hash over the 
references schema objects. This would make rolling upgrades more robust.
The thing is that a statement can potentially reference UDFs which requires 
UDTs, which makes it difficult. OTOH the schema version would invalidate all 
pstsmts - even those that are not affected.
Probably it's ok to just include the hash of the table instead of all 
referenced objects. Not sure on this.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11750) Offline scrub should not abort when it hits corruption

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289741#comment-15289741
 ] 

Yuki Morishita commented on CASSANDRA-11750:


So, to be clear, the issue happens when one of {{system}} tables is corrupted.
In the description above, OP tried to scrub {{system.compactions_in_progress}} 
table, but the actual exception happened during loading schema (this opens all 
system tables) not during scrubbing SSTables.

If {{system}} tables are fine, then scrubbing continues to work in 2.1/2.2.
In 3.0 and above, schema moved to its own keyspace, so in those version if 
schema SSTables are ok then you can scrub system keyspace.

Probably backporting CASSANDRA-11578 to 2.1 and 2.2 (and even 3.0) should do 
the job.


> Offline scrub should not abort when it hits corruption
> --
>
> Key: CASSANDRA-11750
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11750
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Adam Hattrell
>Assignee: Yuki Morishita
>Priority: Minor
>  Labels: Tools
>
> Hit a failure on startup due to corruption of some sstables in system 
> keyspace.  Deleted the listed file and restarted - came down again with 
> another file.
> Figured that I may as well run scrub to clean up all the files.  Got 
> following error:
> {noformat}
> sstablescrub system compaction_history 
> ERROR 17:21:34 Exiting forcefully due to file system exception on startup, 
> disk failure policy "stop" 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/system-compaction_history-ka-1936-CompressionInfo.db
>  
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:131)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at 
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:169)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:741) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:692) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:480) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:376) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:523) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79] 
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_79] 
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79] 
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79] 
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] 
> Caused by: java.io.EOFException: null 
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340) 
> ~[na:1.7.0_79] 
> at java.io.DataInputStream.readUTF(DataInputStream.java:589) ~[na:1.7.0_79] 
> at java.io.DataInputStream.readUTF(DataInputStream.java:564) ~[na:1.7.0_79] 
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:106)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] 
> ... 14 common frames omitted 
> {noformat}
> I guess it might be by design - but I'd argue that I should at least have the 
> option to continue and let it do it's thing.  I'd prefer that sstablescrub 
> ignored the disk failure policy.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11760) dtest failure in TestCQLNodes3RF3_Upgrade_current_2_2_x_To_next_3_x.more_user_types_test

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289730#comment-15289730
 ] 

Tyler Hobbs commented on CASSANDRA-11760:
-

I believe this has the same cause as CASSANDRA-11613, and I'm unable to 
reproduce either error with that fix applied.  [~philipthompson] do you want to 
double check that this is resolved by 11613?

> dtest failure in 
> TestCQLNodes3RF3_Upgrade_current_2_2_x_To_next_3_x.more_user_types_test
> 
>
> Key: CASSANDRA-11760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11760
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.6
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> example failure:
> http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_2_2_x_To_next_3_x/user_types_test/
> I've attached the logs. The test upgrades from 2.2.5 to 3.6. The relevant 
> failure stack trace extracted here:
> {code}
> ERROR [MessagingService-Incoming-/127.0.0.1] 2016-05-11 17:08:31,33
> 4 CassandraDaemon.java:185 - Exception in thread Thread[MessagingSe
> rvice-Incoming-/127.0.0.1,5,main]
> java.lang.ArrayIndexOutOfBoundsException: 1
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:99)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:366)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.composites.AbstractCellNameType$5.deserialize(AbstractCellNameType.java:117)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.composites.AbstractCellNameType$5.deserialize(AbstractCellNameType.java:109)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:106)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:101)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:109)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:322)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:302)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:330)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:272)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99) 
> ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:200)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:177)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:91)
>  ~[apache-cassandra-2.2.6.jar:2.2.6]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11613) dtest failure in upgrade_tests.cql_tests.TestCQLNodes2RF1_2_2_HEAD_UpTo_Trunk.more_user_types_test

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs resolved CASSANDRA-11613.
-
   Resolution: Fixed
Reproduced In:   (was: 3.6)

Thanks, committed with the nits fixed as 
{{dee84ccc59bc5ce76e568bf94ab70c772c079041}} to 3.6 and merged up to 3.7 and 
trunk.

> dtest failure in 
> upgrade_tests.cql_tests.TestCQLNodes2RF1_2_2_HEAD_UpTo_Trunk.more_user_types_test
> --
>
> Key: CASSANDRA-11613
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11613
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.6
>
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all-custom_branch_runs/8/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_2_2_HEAD_UpTo_Trunk/more_user_types_test
> Failed on CassCI build upgrade_tests-all-custom_branch_runs #8



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Freeze UDTs and nested UDTs/collections during schema upgrade

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 0f483d8ff -> 5bdf3184b


Freeze UDTs and nested UDTs/collections during schema upgrade

Patch by Tyler Hobbs; reviewed by Benjamin Lerer for CASSANDRA-11613


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dee84ccc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dee84ccc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dee84ccc

Branch: refs/heads/trunk
Commit: dee84ccc59bc5ce76e568bf94ab70c772c079041
Parents: c17cbe1
Author: Tyler Hobbs 
Authored: Wed May 18 15:15:35 2016 -0500
Committer: Tyler Hobbs 
Committed: Wed May 18 15:15:35 2016 -0500

--
 CHANGES.txt|  2 ++
 .../org/apache/cassandra/db/marshal/AbstractType.java  |  8 
 src/java/org/apache/cassandra/db/marshal/ListType.java | 11 +++
 src/java/org/apache/cassandra/db/marshal/MapType.java  | 13 -
 src/java/org/apache/cassandra/db/marshal/SetType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/UserType.java |  5 -
 .../apache/cassandra/schema/LegacySchemaMigrator.java  |  9 ++---
 7 files changed, 38 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1690e09..a2b472a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.6
+ * Correctly migrate schema for frozen UDTs during 2.x -> 3.x upgrades
+   (does not affect any released versions) (CASSANDRA-11613)
  * Allow server startup if JMX is configured directly (CASSANDRA-11725)
  * Prevent direct memory OOM on buffer pool allocations (CASSANDRA-11710)
  * Enhanced Compaction Logging (CASSANDRA-10805)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/AbstractType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java 
b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
index fc4ec3b..af2cc07 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
@@ -333,14 +333,14 @@ public abstract class AbstractType implements 
Comparator
 }
 
 /**
- * Returns an AbstractType instance that is equivalent to this one, but 
with all nested UDTs explicitly frozen and
- * all collections in UDTs explicitly frozen.
+ * Returns an AbstractType instance that is equivalent to this one, but 
with all nested UDTs and collections
+ * explicitly frozen.
  *
  * This is only necessary for {@code 2.x -> 3.x} schema migrations, and 
can be removed in Cassandra 4.0.
  *
- * See CASSANDRA-11609
+ * See CASSANDRA-11609 and CASSANDRA-11613.
  */
-public AbstractType freezeNestedUDTs()
+public AbstractType freezeNestedMulticellTypes()
 {
 return this;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/ListType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/ListType.java 
b/src/java/org/apache/cassandra/db/marshal/ListType.java
index b3fc4f5..ed843b1 100644
--- a/src/java/org/apache/cassandra/db/marshal/ListType.java
+++ b/src/java/org/apache/cassandra/db/marshal/ListType.java
@@ -110,12 +110,15 @@ public class ListType extends CollectionType
 }
 
 @Override
-public AbstractType freezeNestedUDTs()
+public AbstractType freezeNestedMulticellTypes()
 {
-if (elements.isUDT() && elements.isMultiCell())
+if (!isMultiCell())
+return this;
+
+if (elements.isFreezable() && elements.isMultiCell())
 return getInstance(elements.freeze(), isMultiCell);
-else
-return getInstance(elements.freezeNestedUDTs(), isMultiCell);
+
+return getInstance(elements.freezeNestedMulticellTypes(), isMultiCell);
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/MapType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/MapType.java 
b/src/java/org/apache/cassandra/db/marshal/MapType.java
index 3c5af99..d5cf959 100644
--- a/src/java/org/apache/cassandra/db/marshal/MapType.java
+++ b/src/java/org/apache/cassandra/db/marshal/MapType.java
@@ -117,15 +117,18 @@ public class MapType extends CollectionType>
 }
 
 @Override
-public 

[3/3] cassandra git commit: Merge branch 'cassandra-3.7' into trunk

[tylerhobbs]

Merge branch 'cassandra-3.7' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bdf3184
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bdf3184
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bdf3184

Branch: refs/heads/trunk
Commit: 5bdf3184bdcba969a67f8b5093bdc1a4657cc3c2
Parents: 0f483d8 424b0b9
Author: Tyler Hobbs 
Authored: Wed May 18 15:16:23 2016 -0500
Committer: Tyler Hobbs 
Committed: Wed May 18 15:16:23 2016 -0500

--
 CHANGES.txt|  2 ++
 .../org/apache/cassandra/db/marshal/AbstractType.java  |  8 
 src/java/org/apache/cassandra/db/marshal/ListType.java | 11 +++
 src/java/org/apache/cassandra/db/marshal/MapType.java  | 13 -
 src/java/org/apache/cassandra/db/marshal/SetType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/UserType.java |  5 -
 .../apache/cassandra/schema/LegacySchemaMigrator.java  |  9 ++---
 7 files changed, 38 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bdf3184/CHANGES.txt
--
diff --cc CHANGES.txt
index 4ace3a7,d029c7b..adadefd
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -18,8 -9,9 +18,10 @@@ Merged from 2.2
   * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395)
   * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
  
 +
  3.6
+  * Correctly migrate schema for frozen UDTs during 2.x -> 3.x upgrades
+(does not affect any released versions) (CASSANDRA-11613)
   * Allow server startup if JMX is configured directly (CASSANDRA-11725)
   * Prevent direct memory OOM on buffer pool allocations (CASSANDRA-11710)
   * Enhanced Compaction Logging (CASSANDRA-10805)

[2/3] cassandra git commit: Merge branch 'cassandra-3.6' into cassandra-3.7

[tylerhobbs]

Merge branch 'cassandra-3.6' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/424b0b9e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/424b0b9e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/424b0b9e

Branch: refs/heads/trunk
Commit: 424b0b9e00a700dfdbde6f443dd974d5fbd3e8f7
Parents: ad54683 dee84cc
Author: Tyler Hobbs 
Authored: Wed May 18 15:16:12 2016 -0500
Committer: Tyler Hobbs 
Committed: Wed May 18 15:16:12 2016 -0500

--
 CHANGES.txt|  2 ++
 .../org/apache/cassandra/db/marshal/AbstractType.java  |  8 
 src/java/org/apache/cassandra/db/marshal/ListType.java | 11 +++
 src/java/org/apache/cassandra/db/marshal/MapType.java  | 13 -
 src/java/org/apache/cassandra/db/marshal/SetType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/UserType.java |  5 -
 .../apache/cassandra/schema/LegacySchemaMigrator.java  |  9 ++---
 7 files changed, 38 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/424b0b9e/CHANGES.txt
--
diff --cc CHANGES.txt
index 26c2939,a2b472a..d029c7b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,6 +1,17 @@@
 +3.7
 +Merged from 3.0:
 + * Allow compaction strategies to disable early open (CASSANDRA-11754)
 + * Refactor Materialized View code (CASSANDRA-11475)
 + * Update Java Driver (CASSANDRA-11615)
 +Merged from 2.2:
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 +
  3.6
+  * Correctly migrate schema for frozen UDTs during 2.x -> 3.x upgrades
+(does not affect any released versions) (CASSANDRA-11613)
   * Allow server startup if JMX is configured directly (CASSANDRA-11725)
   * Prevent direct memory OOM on buffer pool allocations (CASSANDRA-11710)
   * Enhanced Compaction Logging (CASSANDRA-10805)

[1/2] cassandra git commit: Freeze UDTs and nested UDTs/collections during schema upgrade

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.7 ad546832f -> 424b0b9e0


Freeze UDTs and nested UDTs/collections during schema upgrade

Patch by Tyler Hobbs; reviewed by Benjamin Lerer for CASSANDRA-11613


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dee84ccc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dee84ccc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dee84ccc

Branch: refs/heads/cassandra-3.7
Commit: dee84ccc59bc5ce76e568bf94ab70c772c079041
Parents: c17cbe1
Author: Tyler Hobbs 
Authored: Wed May 18 15:15:35 2016 -0500
Committer: Tyler Hobbs 
Committed: Wed May 18 15:15:35 2016 -0500

--
 CHANGES.txt|  2 ++
 .../org/apache/cassandra/db/marshal/AbstractType.java  |  8 
 src/java/org/apache/cassandra/db/marshal/ListType.java | 11 +++
 src/java/org/apache/cassandra/db/marshal/MapType.java  | 13 -
 src/java/org/apache/cassandra/db/marshal/SetType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/UserType.java |  5 -
 .../apache/cassandra/schema/LegacySchemaMigrator.java  |  9 ++---
 7 files changed, 38 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1690e09..a2b472a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.6
+ * Correctly migrate schema for frozen UDTs during 2.x -> 3.x upgrades
+   (does not affect any released versions) (CASSANDRA-11613)
  * Allow server startup if JMX is configured directly (CASSANDRA-11725)
  * Prevent direct memory OOM on buffer pool allocations (CASSANDRA-11710)
  * Enhanced Compaction Logging (CASSANDRA-10805)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/AbstractType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java 
b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
index fc4ec3b..af2cc07 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
@@ -333,14 +333,14 @@ public abstract class AbstractType implements 
Comparator
 }
 
 /**
- * Returns an AbstractType instance that is equivalent to this one, but 
with all nested UDTs explicitly frozen and
- * all collections in UDTs explicitly frozen.
+ * Returns an AbstractType instance that is equivalent to this one, but 
with all nested UDTs and collections
+ * explicitly frozen.
  *
  * This is only necessary for {@code 2.x -> 3.x} schema migrations, and 
can be removed in Cassandra 4.0.
  *
- * See CASSANDRA-11609
+ * See CASSANDRA-11609 and CASSANDRA-11613.
  */
-public AbstractType freezeNestedUDTs()
+public AbstractType freezeNestedMulticellTypes()
 {
 return this;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/ListType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/ListType.java 
b/src/java/org/apache/cassandra/db/marshal/ListType.java
index b3fc4f5..ed843b1 100644
--- a/src/java/org/apache/cassandra/db/marshal/ListType.java
+++ b/src/java/org/apache/cassandra/db/marshal/ListType.java
@@ -110,12 +110,15 @@ public class ListType extends CollectionType
 }
 
 @Override
-public AbstractType freezeNestedUDTs()
+public AbstractType freezeNestedMulticellTypes()
 {
-if (elements.isUDT() && elements.isMultiCell())
+if (!isMultiCell())
+return this;
+
+if (elements.isFreezable() && elements.isMultiCell())
 return getInstance(elements.freeze(), isMultiCell);
-else
-return getInstance(elements.freezeNestedUDTs(), isMultiCell);
+
+return getInstance(elements.freezeNestedMulticellTypes(), isMultiCell);
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/MapType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/MapType.java 
b/src/java/org/apache/cassandra/db/marshal/MapType.java
index 3c5af99..d5cf959 100644
--- a/src/java/org/apache/cassandra/db/marshal/MapType.java
+++ b/src/java/org/apache/cassandra/db/marshal/MapType.java
@@ -117,15 +117,18 @@ public class MapType extends CollectionType>
 }
 
 @Override
-

[2/2] cassandra git commit: Merge branch 'cassandra-3.6' into cassandra-3.7

[tylerhobbs]

Merge branch 'cassandra-3.6' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/424b0b9e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/424b0b9e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/424b0b9e

Branch: refs/heads/cassandra-3.7
Commit: 424b0b9e00a700dfdbde6f443dd974d5fbd3e8f7
Parents: ad54683 dee84cc
Author: Tyler Hobbs 
Authored: Wed May 18 15:16:12 2016 -0500
Committer: Tyler Hobbs 
Committed: Wed May 18 15:16:12 2016 -0500

--
 CHANGES.txt|  2 ++
 .../org/apache/cassandra/db/marshal/AbstractType.java  |  8 
 src/java/org/apache/cassandra/db/marshal/ListType.java | 11 +++
 src/java/org/apache/cassandra/db/marshal/MapType.java  | 13 -
 src/java/org/apache/cassandra/db/marshal/SetType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/UserType.java |  5 -
 .../apache/cassandra/schema/LegacySchemaMigrator.java  |  9 ++---
 7 files changed, 38 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/424b0b9e/CHANGES.txt
--
diff --cc CHANGES.txt
index 26c2939,a2b472a..d029c7b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,6 +1,17 @@@
 +3.7
 +Merged from 3.0:
 + * Allow compaction strategies to disable early open (CASSANDRA-11754)
 + * Refactor Materialized View code (CASSANDRA-11475)
 + * Update Java Driver (CASSANDRA-11615)
 +Merged from 2.2:
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 +
  3.6
+  * Correctly migrate schema for frozen UDTs during 2.x -> 3.x upgrades
+(does not affect any released versions) (CASSANDRA-11613)
   * Allow server startup if JMX is configured directly (CASSANDRA-11725)
   * Prevent direct memory OOM on buffer pool allocations (CASSANDRA-11710)
   * Enhanced Compaction Logging (CASSANDRA-10805)

cassandra git commit: Freeze UDTs and nested UDTs/collections during schema upgrade

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.6 c17cbe187 -> dee84ccc5


Freeze UDTs and nested UDTs/collections during schema upgrade

Patch by Tyler Hobbs; reviewed by Benjamin Lerer for CASSANDRA-11613


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dee84ccc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dee84ccc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dee84ccc

Branch: refs/heads/cassandra-3.6
Commit: dee84ccc59bc5ce76e568bf94ab70c772c079041
Parents: c17cbe1
Author: Tyler Hobbs 
Authored: Wed May 18 15:15:35 2016 -0500
Committer: Tyler Hobbs 
Committed: Wed May 18 15:15:35 2016 -0500

--
 CHANGES.txt|  2 ++
 .../org/apache/cassandra/db/marshal/AbstractType.java  |  8 
 src/java/org/apache/cassandra/db/marshal/ListType.java | 11 +++
 src/java/org/apache/cassandra/db/marshal/MapType.java  | 13 -
 src/java/org/apache/cassandra/db/marshal/SetType.java  | 11 +++
 src/java/org/apache/cassandra/db/marshal/UserType.java |  5 -
 .../apache/cassandra/schema/LegacySchemaMigrator.java  |  9 ++---
 7 files changed, 38 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1690e09..a2b472a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.6
+ * Correctly migrate schema for frozen UDTs during 2.x -> 3.x upgrades
+   (does not affect any released versions) (CASSANDRA-11613)
  * Allow server startup if JMX is configured directly (CASSANDRA-11725)
  * Prevent direct memory OOM on buffer pool allocations (CASSANDRA-11710)
  * Enhanced Compaction Logging (CASSANDRA-10805)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/AbstractType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java 
b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
index fc4ec3b..af2cc07 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
@@ -333,14 +333,14 @@ public abstract class AbstractType implements 
Comparator
 }
 
 /**
- * Returns an AbstractType instance that is equivalent to this one, but 
with all nested UDTs explicitly frozen and
- * all collections in UDTs explicitly frozen.
+ * Returns an AbstractType instance that is equivalent to this one, but 
with all nested UDTs and collections
+ * explicitly frozen.
  *
  * This is only necessary for {@code 2.x -> 3.x} schema migrations, and 
can be removed in Cassandra 4.0.
  *
- * See CASSANDRA-11609
+ * See CASSANDRA-11609 and CASSANDRA-11613.
  */
-public AbstractType freezeNestedUDTs()
+public AbstractType freezeNestedMulticellTypes()
 {
 return this;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/ListType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/ListType.java 
b/src/java/org/apache/cassandra/db/marshal/ListType.java
index b3fc4f5..ed843b1 100644
--- a/src/java/org/apache/cassandra/db/marshal/ListType.java
+++ b/src/java/org/apache/cassandra/db/marshal/ListType.java
@@ -110,12 +110,15 @@ public class ListType extends CollectionType
 }
 
 @Override
-public AbstractType freezeNestedUDTs()
+public AbstractType freezeNestedMulticellTypes()
 {
-if (elements.isUDT() && elements.isMultiCell())
+if (!isMultiCell())
+return this;
+
+if (elements.isFreezable() && elements.isMultiCell())
 return getInstance(elements.freeze(), isMultiCell);
-else
-return getInstance(elements.freezeNestedUDTs(), isMultiCell);
+
+return getInstance(elements.freezeNestedMulticellTypes(), isMultiCell);
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dee84ccc/src/java/org/apache/cassandra/db/marshal/MapType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/MapType.java 
b/src/java/org/apache/cassandra/db/marshal/MapType.java
index 3c5af99..d5cf959 100644
--- a/src/java/org/apache/cassandra/db/marshal/MapType.java
+++ b/src/java/org/apache/cassandra/db/marshal/MapType.java
@@ -117,15 +117,18 @@ public class MapType extends CollectionType>
 }
 
 @Override
-

[jira] [Commented] (CASSANDRA-11738) Re-think the use of Severity in the DynamicEndpointSnitch calculation

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289722#comment-15289722
 ] 

Robert Stupp commented on CASSANDRA-11738:
--

Could the sum of all pending+active requests be a good "load" for dsnitch?

> Re-think the use of Severity in the DynamicEndpointSnitch calculation
> -
>
> Key: CASSANDRA-11738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11738
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jeremiah Jordan
> Fix For: 3.x
>
>
> CASSANDRA-11737 was opened to allow completely disabling the use of severity 
> in the DynamicEndpointSnitch calculation, but that is a pretty big hammer.  
> There is probably something we can do to better use the score.
> The issue seems to be that severity is given equal weight with latency in the 
> current code, also that severity is only based on disk io.  If you have a 
> node that is CPU bound on something (say catching up on LCS compactions 
> because of bootstrap/repair/replace) the IO wait can be low, but the latency 
> to the node is high.
> Some ideas I had are:
> 1. Allowing a yaml parameter to tune how much impact the severity score has 
> in the calculation.
> 2. Taking CPU load into account as well as IO Wait (this would probably help 
> in the cases I have seen things go sideways)
> 3. Move the -D from CASSANDRA-11737 to being a yaml level setting
> 4. Go back to just relying on Latency and get rid of severity all together.  
> Now that we have rapid read protection, maybe just using latency is enough, 
> as it can help where the predictive nature of IO wait would have been useful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11837) dtest failure in topology_test.TestTopology.simple_decommission_test

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289714#comment-15289714
 ] 

Philip Thompson commented on CASSANDRA-11837:
-

Testing a fix here: 
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/103/

> dtest failure in topology_test.TestTopology.simple_decommission_test
> 
>
> Key: CASSANDRA-11837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11837
> Project: Cassandra
>  Issue Type: Test
>Reporter: Philip Thompson
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1223/testReport/topology_test/TestTopology/simple_decommission_test
> Failed on CassCI build trunk_dtest #1223
> The problem is that node3 detected node2 as down before the stop call was 
> made, so the wait_other_notice check fails. The fix here is almost certainly 
> as simple as just changing that line to {{node2.stop()}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11837) dtest failure in topology_test.TestTopology.simple_decommission_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson reassigned CASSANDRA-11837:
---

Assignee: Philip Thompson  (was: DS Test Eng)

> dtest failure in topology_test.TestTopology.simple_decommission_test
> 
>
> Key: CASSANDRA-11837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11837
> Project: Cassandra
>  Issue Type: Test
>Reporter: Philip Thompson
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1223/testReport/topology_test/TestTopology/simple_decommission_test
> Failed on CassCI build trunk_dtest #1223
> The problem is that node3 detected node2 as down before the stop call was 
> made, so the wait_other_notice check fails. The fix here is almost certainly 
> as simple as just changing that line to {{node2.stop()}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9669:
---
Status: Ready to Commit  (was: Patch Available)

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11840) Set a more conservative default to streaming_socket_timeout_in_ms

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11840:

Status: Patch Available  (was: Open)

> Set a more conservative default to streaming_socket_timeout_in_ms
> -
>
> Key: CASSANDRA-11840
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11840
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9669:
---
Reproduced In: 2.1.7, 2.0.16, 1.2.19  (was: 1.2.19, 2.0.16, 2.1.7)
 Reviewer: Benedict

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11840) Set a more conservative default to streaming_socket_timeout_in_ms

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289711#comment-15289711
 ] 

Paulo Motta commented on CASSANDRA-11840:
-

Attaching patch that sets default {{streaming_socket_timeout_in_ms}} to 48 
hours (17280ms), so it gives plenty of time to transfer very large sstables 
or rebuild indexes but still will not let streams hang indefinitely before 
CASSANDRA-11841 is in place. Also added more informative message to increase 
{{streaming_socket_timeout_in_ms}} if active stream sessions fail with 
{{SocketTimeoutException}}.

Patch merges cleanly from 2.1 to 2.2, minor conflict on 3.0, then merge cleanly 
upwards.

||2.1||2.2||3.0||3.7||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.1...pauloricardomg:2.1-11839]|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-11839]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-11839]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.7...pauloricardomg:3.7-11839]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-11839]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-11839-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-11839-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-11839-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.7-11839-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-11839-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-11839-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-11839-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-11839-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.7-11839-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-11839-dtest/lastCompletedBuild/testReport/]|

> Set a more conservative default to streaming_socket_timeout_in_ms
> -
>
> Key: CASSANDRA-11840
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11840
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289709#comment-15289709
 ] 

Benedict commented on CASSANDRA-9669:
-

Sure; +1

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289708#comment-15289708
 ] 

Tyler Hobbs commented on CASSANDRA-10786:
-

I agree that it's best to change this behavior as part of the v5 protocol.

bq. The downside of tying this change to a protocol version bump is that I 
imagine it defers this change quite a bit, and this fix would be really nice to 
have as it will fix a problem a lot of users have run into.

Not necessarily.  We haven't needed to release a v5 protocol yet, but I don't 
believe anybody would object to including it in a 3.x release.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9669:
---
Reviewer:   (was: Branimir Lambov)

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-10786:

Labels: client-impacting protocolv5  (was: client-impacting)

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11731) dtest failure in pushed_notifications_test.TestPushedNotifications.move_single_node_test

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289703#comment-15289703
 ] 

Philip Thompson commented on CASSANDRA-11731:
-

[~stefania], you worked on CASSANDRA-8516, could the issue we're seeing here be 
a bug? Looking at this debug output, 
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/99/testReport/node_0_iter_017.pushed_notifications_test/TestPushedNotifications/move_single_node_test/

We have waited over a minute after the cluster has started up, and cleared all 
notifications. Then when we move node1, node2 sends a NEW_NODE and a MOVED_NODE 
notification. It is clearly happening after the move, and not being left over 
from beforehand. Either this is an incredibly delayed notification, or an extra 
notification is being sent erroneously.

> dtest failure in 
> pushed_notifications_test.TestPushedNotifications.move_single_node_test
> 
>
> Key: CASSANDRA-11731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11731
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> one recent failure (no vnode job)
> {noformat}
> 'MOVED_NODE' != u'NEW_NODE'
> {noformat}
> http://cassci.datastax.com/job/trunk_novnode_dtest/366/testReport/pushed_notifications_test/TestPushedNotifications/move_single_node_test
> Failed on CassCI build trunk_novnode_dtest #366



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289699#comment-15289699
 ] 

Joshua McKenzie commented on CASSANDRA-9669:


  [~benedict]: Care to take review of these small fix-up patches?

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8343) Secondary index creation causes moves/bootstraps to fail

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-8343.

Resolution: Fixed

> Secondary index creation causes moves/bootstraps to fail
> 
>
> Key: CASSANDRA-8343
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8343
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Frisch
>Assignee: Paulo Motta
>
> Node moves/bootstraps are failing if the stream timeout is set to a value in 
> which secondary index creation cannot complete.  This happens because at the 
> end of the very last stream the StreamInSession.closeIfFinished() function 
> calls maybeBuildSecondaryIndexes on every column family.  If the stream time 
> + all CF's index creation takes longer than your stream timeout then the 
> socket closes from the sender's side, the receiver of the stream tries to 
> write to said socket because it's not null, an IOException is thrown but not 
> caught in closeIfFinished(), the exception is caught somewhere and not 
> logged, AbstractStreamSession.close() is never called, and the CountDownLatch 
> is never decremented.  This causes the move/bootstrap to continue forever 
> until the node is restarted.
> This problem of stream time + secondary index creation time exists on 
> decommissioning/unbootstrap as well but since it's on the sending side the 
> timeout triggers the onFailure() callback which does decrement the 
> CountDownLatch leading to completion.
> A cursory glance at the 2.0 code leads me to believe this problem would exist 
> there as well.
> Temporary workaround: set a really high/infinite stream timeout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9669) If sstable flushes complete out of order, on restart we can fail to replay necessary commit log records

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9669:
---
Assignee: Branimir Lambov  (was: Benedict)

> If sstable flushes complete out of order, on restart we can fail to replay 
> necessary commit log records
> ---
>
> Key: CASSANDRA-9669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9669
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Benedict
>Assignee: Branimir Lambov
>Priority: Critical
>  Labels: correctness
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> While {{postFlushExecutor}} ensures it never expires CL entries out-of-order, 
> on restart we simply take the maximum replay position of any sstable on disk, 
> and ignore anything prior. 
> It is quite possible for there to be two flushes triggered for a given table, 
> and for the second to finish first by virtue of containing a much smaller 
> quantity of live data (or perhaps the disk is just under less pressure). If 
> we crash before the first sstable has been written, then on restart the data 
> it would have represented will disappear, since we will not replay the CL 
> records.
> This looks to be a bug present since time immemorial, and also seems pretty 
> serious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11483) Enhance sstablemetadata

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289694#comment-15289694
 ] 

Jeremiah Jordan commented on CASSANDRA-11483:
-

+1 for defaulting the colors and unicode to off.

> Enhance sstablemetadata
> ---
>
> Key: CASSANDRA-11483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11483
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 3.x
>
> Attachments: CASSANDRA-11483.txt, CASSANDRA-11483v2.txt, 
> CASSANDRA-11483v3.txt, CASSANDRA-11483v4.txt, Screen Shot 2016-04-03 at 
> 11.40.32 PM.png
>
>
> sstablemetadata provides quite a bit of useful information but theres a few 
> hiccups I would like to see addressed:
> * Does not use client mode
> * Units are not provided (or anything for that matter). There is data in 
> micros, millis, seconds as durations and timestamps from epoch. But there is 
> no way to tell what one is without a non-trival code dive
> * in general pretty frustrating to parse



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11839) Active streams fail with SocketTimeoutException

[Paulo Motta (JIRA)]

Paulo Motta created CASSANDRA-11839:
---

 Summary: Active streams fail with SocketTimeoutException
 Key: CASSANDRA-11839
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11839
 Project: Cassandra
  Issue Type: Bug
Reporter: Paulo Motta
Assignee: Paulo Motta


The original reasoning behind {{streaming_socket_timeout_in_ms}} was to kill 
one-sided hanging streams (CASSANDRA-3838). This was never much of a problem 
when the default was zero (never timeout).

On CASSANDRA-8611 we changed the default to 1 hour, but it was never enforced 
due to CASSANDRA-11286, which was fixed recently.

On recent releases we've been receiving reports of stream failures when 
streaming large files, because the sender incoming socket becomes inactive, 
times out after 1 hour, and the stream session fails with 
{{SocketTimeoutException}} (CASSANDRA-11345, CASSANDRA-11826), even though the 
stream session is still active. The session also fails if 2i/MV rebuild takes 
longer than 1 hour on the receiver (CASSANDRA-8343).

The definitive fix on trunk is to add a {{KeepAlive}} message to the stream 
protocol to detect broken connections and retire 
{{streaming_socket_timeout_in_ms}}. But we must also increase the default 
{{streaming_socket_timeout_in_ms}} in older versions to a more conservative 
value, so it is still able to detect long hanging streams.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289691#comment-15289691
 ] 

Robert Stupp commented on CASSANDRA-10786:
--

I'm +1 on this with a new protocol version (or option) for the mentioned 
reasons plus it's hard to explain to people. Happy if this goes in.
But I'm strongly (non-binding) -1 to change it in the current or previous 
protocol versions.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11840) Set a more conservative default to streaming_socket_timeout_in_ms

[Paulo Motta (JIRA)]

Paulo Motta created CASSANDRA-11840:
---

 Summary: Set a more conservative default to 
streaming_socket_timeout_in_ms
 Key: CASSANDRA-11840
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11840
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Paulo Motta
Assignee: Paulo Motta






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11841) Add keep-alive to stream protocol

[Paulo Motta (JIRA)]

Paulo Motta created CASSANDRA-11841:
---

 Summary: Add keep-alive to stream protocol
 Key: CASSANDRA-11841
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11841
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Paulo Motta
Assignee: Paulo Motta






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11826) Repairs failing for some of bigger tables

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289675#comment-15289675
 ] 

Paulo Motta commented on CASSANDRA-11826:
-

Thanks for the report. Will close this as duplicate of CASSANDRA-11839 which 
should be resolved soon.

Temporary workaround for this is to increase {{streaming_socket_timeout_in_ms}}.

> Repairs failing for some of bigger tables
> -
>
> Key: CASSANDRA-11826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11826
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Centos 6
>Reporter: vin01
>Priority: Minor
>
> Architecture for the same :-
> Cassandra version 2.2.4
> 3 Nodes - Node-1, Node-2, Node-3, repair started on Node-1 and following logs 
> are also from Node-1 itself.
> Node-2 and Node-3 are in same data-center, and Node-1 is in a different 
> data-center. 
> Value of streaming_socket_timeout_in_ms is set to 1 Hour which is default for 
> version 2.2.4. 
> Repair succeeded for some of table but failed for one of tables which is 
> quite huge, i looked at sizes of .db files in data directory and there are 
> files up to 1.5 GB for that table.
> 1.4G  la-59543-big-Data.db
> Logs for the same when repair failed :-
> WARN  [RepairJobTask:5] 2016-05-18 02:25:31,137 RepairJob.java:162 - [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd] TABLE_NAME sync failed
> ERROR [RepairJobTask:5] 2016-05-18 02:25:31,138 RepairSession.java:290 - 
> [repair #5a855825-1cb7-11e6-9f5f-b717b380ffdd] Session completed with the 
> following error
> org.apache.cassandra.exceptions.RepairException: [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd on KEYSPACE_NAME/TABLE_NAME, 
> (-7892648649079895028,-7818964903881740266]] Sync failed between /Node-3 and 
> /Node-2
> at 
> org.apache.cassandra.repair.RemoteSyncTask.syncComplete(RemoteSyncTask.java:67)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:211)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:415)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_72]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_72]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
> ERROR [RepairJobTask:5] 2016-05-18 02:25:31,138 RepairRunnable.java:243 - 
> Repair session 5a855825-1cb7-11e6-9f5f-b717b380ffdd for range 
> (-7892648649079895028,-7818964903881740266] failed with error [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd on KEYSPACE_NA$
> org.apache.cassandra.exceptions.RepairException: [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd on KEYSPACE_NAME/TABLE_NAME, 
> (-7892648649079895028,-7818964903881740266]] Sync failed between /Node-3 and 
> /Node-2
> at 
> org.apache.cassandra.repair.RemoteSyncTask.syncComplete(RemoteSyncTask.java:67)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:211)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:415)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_72]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_72]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
> ERROR [STREAM-IN-/Node-2] 2016-05-18 02:25:33,296 StreamSession.java:524 - 
> [Stream #efe521b0-1cb8-11e6-9f5f-b717b380ffdd] Streaming error occurred
> java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) 
> ~[na:1.8.0_72]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
> ~[na:1.8.0_72]
> at java.net.SocketInputStream.read(SocketInputStream.java:170) 
> ~[na:1.8.0_72]
> at 

[jira] [Commented] (CASSANDRA-8343) Secondary index creation causes moves/bootstraps to fail

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289672#comment-15289672
 ] 

Paulo Motta commented on CASSANDRA-8343:


Closing as duplicate of more general ticket CASSANDRA-11839.

> Secondary index creation causes moves/bootstraps to fail
> 
>
> Key: CASSANDRA-8343
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8343
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Frisch
>Assignee: Paulo Motta
>
> Node moves/bootstraps are failing if the stream timeout is set to a value in 
> which secondary index creation cannot complete.  This happens because at the 
> end of the very last stream the StreamInSession.closeIfFinished() function 
> calls maybeBuildSecondaryIndexes on every column family.  If the stream time 
> + all CF's index creation takes longer than your stream timeout then the 
> socket closes from the sender's side, the receiver of the stream tries to 
> write to said socket because it's not null, an IOException is thrown but not 
> caught in closeIfFinished(), the exception is caught somewhere and not 
> logged, AbstractStreamSession.close() is never called, and the CountDownLatch 
> is never decremented.  This causes the move/bootstrap to continue forever 
> until the node is restarted.
> This problem of stream time + secondary index creation time exists on 
> decommissioning/unbootstrap as well but since it's on the sending side the 
> timeout triggers the onFailure() callback which does decrement the 
> CountDownLatch leading to completion.
> A cursory glance at the 2.0 code leads me to believe this problem would exist 
> there as well.
> Temporary workaround: set a really high/infinite stream timeout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11826) Repairs failing for some of bigger tables

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-11826.
-
Resolution: Duplicate

> Repairs failing for some of bigger tables
> -
>
> Key: CASSANDRA-11826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11826
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Centos 6
>Reporter: vin01
>Priority: Minor
>
> Architecture for the same :-
> Cassandra version 2.2.4
> 3 Nodes - Node-1, Node-2, Node-3, repair started on Node-1 and following logs 
> are also from Node-1 itself.
> Node-2 and Node-3 are in same data-center, and Node-1 is in a different 
> data-center. 
> Value of streaming_socket_timeout_in_ms is set to 1 Hour which is default for 
> version 2.2.4. 
> Repair succeeded for some of table but failed for one of tables which is 
> quite huge, i looked at sizes of .db files in data directory and there are 
> files up to 1.5 GB for that table.
> 1.4G  la-59543-big-Data.db
> Logs for the same when repair failed :-
> WARN  [RepairJobTask:5] 2016-05-18 02:25:31,137 RepairJob.java:162 - [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd] TABLE_NAME sync failed
> ERROR [RepairJobTask:5] 2016-05-18 02:25:31,138 RepairSession.java:290 - 
> [repair #5a855825-1cb7-11e6-9f5f-b717b380ffdd] Session completed with the 
> following error
> org.apache.cassandra.exceptions.RepairException: [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd on KEYSPACE_NAME/TABLE_NAME, 
> (-7892648649079895028,-7818964903881740266]] Sync failed between /Node-3 and 
> /Node-2
> at 
> org.apache.cassandra.repair.RemoteSyncTask.syncComplete(RemoteSyncTask.java:67)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:211)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:415)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_72]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_72]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
> ERROR [RepairJobTask:5] 2016-05-18 02:25:31,138 RepairRunnable.java:243 - 
> Repair session 5a855825-1cb7-11e6-9f5f-b717b380ffdd for range 
> (-7892648649079895028,-7818964903881740266] failed with error [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd on KEYSPACE_NA$
> org.apache.cassandra.exceptions.RepairException: [repair 
> #5a855825-1cb7-11e6-9f5f-b717b380ffdd on KEYSPACE_NAME/TABLE_NAME, 
> (-7892648649079895028,-7818964903881740266]] Sync failed between /Node-3 and 
> /Node-2
> at 
> org.apache.cassandra.repair.RemoteSyncTask.syncComplete(RemoteSyncTask.java:67)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:211)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:415)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_72]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_72]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
> ERROR [STREAM-IN-/Node-2] 2016-05-18 02:25:33,296 StreamSession.java:524 - 
> [Stream #efe521b0-1cb8-11e6-9f5f-b717b380ffdd] Streaming error occurred
> java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) 
> ~[na:1.8.0_72]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
> ~[na:1.8.0_72]
> at java.net.SocketInputStream.read(SocketInputStream.java:170) 
> ~[na:1.8.0_72]
> at java.net.SocketInputStream.read(SocketInputStream.java:141) 
> ~[na:1.8.0_72]
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) 
> ~[na:1.8.0_72]
> at sun.security.ssl.InputRecord.read(InputRecord.java:503) 

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289652#comment-15289652
 ] 

Alex Petrov commented on CASSANDRA-10786:
-

[~adutra] has also proposed to make a toggle switch for this feature (something 
like {{invalidate_prepare_statements_on_schema_changes}}). Or wait for a 
protocol version bump.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11838) dtest failure in largecolumn_test:TestLargeColumn.cleanup_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11838:

Reproduced In: 3.6

> dtest failure in largecolumn_test:TestLargeColumn.cleanup_test
> --
>
> Key: CASSANDRA-11838
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11838
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log
>
>
> Example failure at:
> http://cassci.datastax.com/job/trunk_offheap_dtest/200/testReport/largecolumn_test/TestLargeColumn/cleanup_test/
> node 1 contains the following OOM in its log:
> {code}
> ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,112 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xb97f2640, 
> L:/127.0.0.1:9042 - R:/127.0.0.1:48190]
> java.lang.OutOfMemoryError: Java heap space
>   at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) 
> ~[main/:na]
>   at 
> org.apache.cassandra.transport.CBUtil.readBoundValue(CBUtil.java:407) 
> ~[main/:na]
>   at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:462) 
> ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:417) 
> ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:365) 
> ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:41)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:280)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:261)
>  ~[main/:na]
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:277)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:264)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:962)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:879)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:360) 
> ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:276) 
> ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,756 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xba0be401, 
> L:/127.0.0.1:9042 - R:/127.0.0.1:48191]
> java.lang.OutOfMemoryError: Java heap space
>   at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) 
> ~[main/:na]
>   at 
> 

[jira] [Updated] (CASSANDRA-11838) dtest failure in largecolumn_test:TestLargeColumn.cleanup_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-11838:

Labels: dtest  (was: )

> dtest failure in largecolumn_test:TestLargeColumn.cleanup_test
> --
>
> Key: CASSANDRA-11838
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11838
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log
>
>
> Example failure at:
> http://cassci.datastax.com/job/trunk_offheap_dtest/200/testReport/largecolumn_test/TestLargeColumn/cleanup_test/
> node 1 contains the following OOM in its log:
> {code}
> ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,112 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xb97f2640, 
> L:/127.0.0.1:9042 - R:/127.0.0.1:48190]
> java.lang.OutOfMemoryError: Java heap space
>   at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) 
> ~[main/:na]
>   at 
> org.apache.cassandra.transport.CBUtil.readBoundValue(CBUtil.java:407) 
> ~[main/:na]
>   at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:462) 
> ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:417) 
> ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:365) 
> ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:41)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:280)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:261)
>  ~[main/:na]
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:277)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:264)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:962)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:879)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:360) 
> ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:276) 
> ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
>  ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,756 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xba0be401, 
> L:/127.0.0.1:9042 - R:/127.0.0.1:48191]
> java.lang.OutOfMemoryError: Java heap space
>   at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) 
> ~[main/:na]
>   at 
> 

[jira] [Created] (CASSANDRA-11838) dtest failure in largecolumn_test:TestLargeColumn.cleanup_test

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-11838:
---

 Summary: dtest failure in 
largecolumn_test:TestLargeColumn.cleanup_test
 Key: CASSANDRA-11838
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11838
 Project: Cassandra
  Issue Type: Bug
Reporter: Philip Thompson
 Fix For: 3.x
 Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log

Example failure at:

http://cassci.datastax.com/job/trunk_offheap_dtest/200/testReport/largecolumn_test/TestLargeColumn/cleanup_test/

node 1 contains the following OOM in its log:

{code}
ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,112 Message.java:611 - 
Unexpected exception during request; channel = [id: 0xb97f2640, 
L:/127.0.0.1:9042 - R:/127.0.0.1:48190]
java.lang.OutOfMemoryError: Java heap space
at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) 
~[main/:na]
at 
org.apache.cassandra.transport.CBUtil.readBoundValue(CBUtil.java:407) 
~[main/:na]
at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:462) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:417) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:365) 
~[main/:na]
at 
org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:45)
 ~[main/:na]
at 
org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:41)
 ~[main/:na]
at 
org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:280) 
~[main/:na]
at 
org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:261) 
~[main/:na]
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:277)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:264)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:962)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:879)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:360) 
~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:276) 
~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at 
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
 ~[netty-all-4.0.36.Final.jar:4.0.36.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,756 Message.java:611 - 
Unexpected exception during request; channel = [id: 0xba0be401, 
L:/127.0.0.1:9042 - R:/127.0.0.1:48191]
java.lang.OutOfMemoryError: Java heap space
at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) 
~[main/:na]
at 
org.apache.cassandra.transport.CBUtil.readBoundValue(CBUtil.java:407) 
~[main/:na]
at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:462) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:417) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:365) 

[jira] [Created] (CASSANDRA-11837) dtest failure in topology_test.TestTopology.simple_decommission_test

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-11837:
---

 Summary: dtest failure in 
topology_test.TestTopology.simple_decommission_test
 Key: CASSANDRA-11837
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11837
 Project: Cassandra
  Issue Type: Test
Reporter: Philip Thompson
Assignee: DS Test Eng


example failure:

http://cassci.datastax.com/job/trunk_dtest/1223/testReport/topology_test/TestTopology/simple_decommission_test

Failed on CassCI build trunk_dtest #1223

The problem is that node3 detected node2 as down before the stop call was made, 
so the wait_other_notice check fails. The fix here is almost certainly as 
simple as just changing that line to {{node2.stop()}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11686) dtest failure in replication_test.SnitchConfigurationUpdateTest.test_rf_expand_gossiping_property_file_snitch_multi_dc

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289436#comment-15289436
 ] 

Philip Thompson commented on CASSANDRA-11686:
-

Looking at the logs for this failure of this test: 
http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/229/testReport/replication_test/SnitchConfigurationUpdateTest/test_rf_expand_gossiping_property_file_snitch_multi_dc/

I see that every node had some sort of 15 second pause, which is hilariously 
long. I expect this needs moved to large_dtest, as I see no other problems.

> dtest failure in 
> replication_test.SnitchConfigurationUpdateTest.test_rf_expand_gossiping_property_file_snitch_multi_dc
> --
>
> Key: CASSANDRA-11686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11686
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Russ Hatch
>  Labels: dtest
>
> intermittent failure. this test also fails on windows but looks to be for 
> another reason (CASSANDRA-11439)
> http://cassci.datastax.com/job/cassandra-3.0_dtest/682/testReport/replication_test/SnitchConfigurationUpdateTest/test_rf_expand_gossiping_property_file_snitch_multi_dc/
> {noformat}
> Nodetool command '/home/automaton/cassandra/bin/nodetool -h localhost -p 7400 
> getendpoints testing rf_test dummy' failed; exit status: 1; stderr: nodetool: 
> Failed to connect to 'localhost:7400' - ConnectException: 'Connection 
> refused'.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11836) dtest failure in repair_tests.repair_test.TestRepair.thread_count_repair_test

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-11836:
---

 Summary: dtest failure in 
repair_tests.repair_test.TestRepair.thread_count_repair_test
 Key: CASSANDRA-11836
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11836
 Project: Cassandra
  Issue Type: Test
Reporter: Philip Thompson
Assignee: DS Test Eng


example failure:

http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/239/testReport/repair_tests.repair_test/TestRepair/thread_count_repair_test

Failed on CassCI build cassandra-3.0_dtest_win32 #239

Error is
{code}
ERROR [RepairTracePolling] 2016-05-18 08:58:10,479 CassandraDaemon.java:195 - 
Exception in thread Thread[RepairTracePolling,5,RMI Runtime]
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
received only 0 responses.
at 
org.apache.cassandra.service.ReadCallback.awaitResults(ReadCallback.java:132) 
~[main/:na]
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:137) 
~[main/:na]
at 
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:145)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.awaitResultsAndRetryOnDigestMismatch(StorageProxy.java:1711)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1660) 
~[main/:na]
at 
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
~[main/:na]
at 
org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
~[main/:na]
at 
org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:954)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:263)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
 ~[main/:na]
at 
org.apache.cassandra.repair.RepairRunnable$4.runMayThrow(RepairRunnable.java:407)
 ~[main/:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[main/:na]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_51]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11835) dtest failure in replace_address_test.TestReplaceAddress.replace_with_reset_resume_state_test

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-11835:
---

 Summary: dtest failure in 
replace_address_test.TestReplaceAddress.replace_with_reset_resume_state_test
 Key: CASSANDRA-11835
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11835
 Project: Cassandra
  Issue Type: Test
Reporter: Philip Thompson
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
node3.log, node3_debug.log, node4.log, node4_debug.log

example failure:

http://cassci.datastax.com/job/cassandra-2.2_offheap_dtest/375/testReport/replace_address_test/TestReplaceAddress/replace_with_reset_resume_state_test

Failed on CassCI build cassandra-2.2_offheap_dtest #375

Node4 is started up to replace the killed node3, but fails with this error:

{code}
ERROR [main] 2016-05-18 03:08:02,244 CassandraDaemon.java:638 - Exception 
encountered during startup
java.lang.RuntimeException: Cannot replace_address /127.0.0.3 because it 
doesn't exist in gossip
at 
org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:529)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:775)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:709) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:595) 
~[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625) 
[main/:na]
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[08/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jbellis]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/958183fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/958183fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/958183fa

Branch: refs/heads/cassandra-3.0
Commit: 958183fadab6a67abf8831fa8e92751039a2ef58
Parents: f73b485 be65393
Author: Jonathan Ellis 
Authored: Wed May 18 12:30:54 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:54 2016 -0500

--

--

[jira] [Assigned] (CASSANDRA-11831) Ability to disable purgeable tombstone check via startup flag

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-11831:
---

Assignee: Marcus Eriksson

> Ability to disable purgeable tombstone check via startup flag
> -
>
> Key: CASSANDRA-11831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11831
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Ryan Svihla
>Assignee: Marcus Eriksson
>
> On Cassandra 2.1.14 hen a node gets way behind and has 10s of thousand 
> sstables it appears a lot of the CPU time is spent doing checks like this on 
> a call to getMaxPurgeableTimestamp 
>  org.apache.cassandra.utils.Murmur3BloomFilter.hash(java.nio.ByteBuffer, 
> int, int, long, long[]) @bci=13, line=57 (Compiled frame; information may be 
> imprecise)
> - org.apache.cassandra.utils.BloomFilter.indexes(java.nio.ByteBuffer) 
> @bci=22, line=82 (Compiled frame)
> - org.apache.cassandra.utils.BloomFilter.isPresent(java.nio.ByteBuffer) 
> @bci=2, line=107 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.CompactionController.maxPurgeableTimestamp(org.apache.cassandra.db.DecoratedKey)
>  @bci=89, line=186 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.getMaxPurgeableTimestamp()
>  @bci=21, line=99 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.access$300(org.apache.cassandra.db.compaction.LazilyCompactedRow)
>  @bci=1, line=49 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.LazilyCompactedRow$Reducer.getReduced() 
> @bci=241, line=296 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.LazilyCompactedRow$Reducer.getReduced() 
> @bci=1, line=206 (Compiled frame)
> - org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext() 
> @bci=44, line=206 (Compiled frame)
> - com.google.common.collect.AbstractIterator.tryToComputeNext() @bci=9, 
> line=143 (Compiled frame)
> - com.google.common.collect.AbstractIterator.hasNext() @bci=61, line=138 
> (Compiled frame)
> - com.google.common.collect.Iterators$7.computeNext() @bci=4, line=645 
> (Compiled frame)
> - com.google.common.collect.AbstractIterator.tryToComputeNext() @bci=9, 
> line=143 (Compiled frame)
> - com.google.common.collect.AbstractIterator.hasNext() @bci=61, line=138 
> (Compiled frame)
> - 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(java.util.Iterator)
>  @bci=1, line=166 (Compiled frame)
> - org.apache.cassandra.db.compaction.LazilyCompactedRow.write(long, 
> org.apache.cassandra.io.util.DataOutputPlus) @bci=52, line=121 (Compiled 
> frame)
> - 
> org.apache.cassandra.io.sstable.SSTableWriter.append(org.apache.cassandra.db.compaction.AbstractCompactedRow)
>  @bci=18, line=193 (Compiled frame)
> - 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(org.apache.cassandra.db.compaction.AbstractCompactedRow)
>  @bci=13, line=127 (Compiled frame)
> - org.apache.cassandra.db.compaction.CompactionTask.runMayThrow() 
> @bci=666, line=197 (Compiled frame)
> - org.apache.cassandra.utils.WrappedRunnable.run() @bci=1, line=28 
> (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(org.apache.cassandra.db.compaction.CompactionManager$CompactionExecutorStatsCollector)
>  @bci=6, line=73 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(org.apache.cassandra.db.compaction.CompactionManager$CompactionExecutorStatsCollector)
>  @bci=2, line=59 (Compiled frame)
> - 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run()
>  @bci=125, line=264 (Compiled frame)
> - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
> (Compiled frame)
> - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Compiled frame)
> - 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
>  @bci=95, line=1142 (Compiled frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
> (Compiled frame)
> - java.lang.Thread.run() @bci=11, line=745 (Compiled frame)
> If we could at least on startup pass a flag like -DskipTombstonePurgeCheck so 
> we could in these particularly bad cases just avoid the calculation and merge 
> tables until we have less to worry about then restart the node with that flag 
> missing once we're down to a more manageable amount of sstables. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11834:
---
   Resolution: Fixed
Fix Version/s: 2.2.7
   Status: Resolved  (was: Patch Available)

committed to 2.1 and 2.2

> Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
> expired tombstone
> ---
>
> Key: CASSANDRA-11834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 2.1.15, 2.2.7
>
> Attachments: 11834.txt
>
>
> In LCR's get reduced, we currently do this:
> {code}
> if (t.timestamp() < getMaxPurgeableTimestamp() && 
> t.data.isGcAble(controller.gcBefore))
> {code}
> Should call the expensive getMaxPurgeableTimestamp only after we have called 
> the cheap isGcAble.
> Marking this as a bug since it can cause pathological performance problems 
> (CASSANDRA-11831).
> Have verified that this is not a problem in 3.0 (CompactionIterator does the 
> check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[07/10] cassandra git commit: merge from 2.1

[jbellis]

merge from 2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be653932
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be653932
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be653932

Branch: refs/heads/trunk
Commit: be653932d883134ec6ebecd5b9ad1dbb64bda64a
Parents: 89344f1 00f2540
Author: Jonathan Ellis 
Authored: Wed May 18 12:28:45 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:14 2016 -0500

--
 CHANGES.txt | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/be653932/CHANGES.txt
--
diff --cc CHANGES.txt
index befb520,3972248..103d057
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,8 +1,24 @@@
-  * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
- 
 -2.1.15
 +2.2.7
++ * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+expired tombstone (CASSANDRA-11834)
 - * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
 -   (CASSANDRA-11679)
   * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
   * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
   * Make cython optional in pylib/setup.py (CASSANDRA-11630)
@@@ -25,62 -11,21 +27,63 @@@
   * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
   * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
   * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
 + * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 +   processes (CASSANDRA-11505)
-  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+sstables (CASSANDRA-9935)
  
  
 -2.1.14
 - * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 -   going (CASSANDRA-10979)
 - * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 -   processes (CASSANDRA-11505)
 - * Replace sstables on DataTracker before marking them as non-compacting 
during anti-compaction (CASSANDRA-11548)
 +2.2.6
 + * Allow only DISTINCT queries with partition keys restrictions 
(CASSANDRA-11339)
 + * CqlConfigHelper no longer requires both a keystore and truststore to work 
(CASSANDRA-11532)
 + * Make deprecated repair methods backward-compatible with previous 
notification service (CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
 + * DatabaseDescriptor should log stacktrace in case of Eception during seed 
provider creation (CASSANDRA-11312)
 + * Use canonical path for directory in SSTable descriptor (CASSANDRA-10587)
 + * Add cassandra-stress keystore option (CASSANDRA-9325)
 + * Fix out-of-space error treatment in memtable flushing (CASSANDRA-11448).
 + * Dont mark sstables as repairing with sub range repairs (CASSANDRA-11451)
 + * Fix use of NullUpdater for 2i during compaction (CASSANDRA-11450)
 + * Notify when sstables change after cancelling compaction (CASSANDRA-11373)
 + * cqlsh: COPY FROM should check that explicit column names are valid 
(CASSANDRA-11333)
 + * Add -Dcassandra.start_gossip startup 

[09/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jbellis]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/958183fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/958183fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/958183fa

Branch: refs/heads/trunk
Commit: 958183fadab6a67abf8831fa8e92751039a2ef58
Parents: f73b485 be65393
Author: Jonathan Ellis 
Authored: Wed May 18 12:30:54 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:54 2016 -0500

--

--

[05/10] cassandra git commit: merge from 2.1

[jbellis]

merge from 2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be653932
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be653932
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be653932

Branch: refs/heads/cassandra-3.0
Commit: be653932d883134ec6ebecd5b9ad1dbb64bda64a
Parents: 89344f1 00f2540
Author: Jonathan Ellis 
Authored: Wed May 18 12:28:45 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:14 2016 -0500

--
 CHANGES.txt | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/be653932/CHANGES.txt
--
diff --cc CHANGES.txt
index befb520,3972248..103d057
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,8 +1,24 @@@
-  * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
- 
 -2.1.15
 +2.2.7
++ * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+expired tombstone (CASSANDRA-11834)
 - * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
 -   (CASSANDRA-11679)
   * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
   * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
   * Make cython optional in pylib/setup.py (CASSANDRA-11630)
@@@ -25,62 -11,21 +27,63 @@@
   * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
   * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
   * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
 + * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 +   processes (CASSANDRA-11505)
-  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+sstables (CASSANDRA-9935)
  
  
 -2.1.14
 - * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 -   going (CASSANDRA-10979)
 - * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 -   processes (CASSANDRA-11505)
 - * Replace sstables on DataTracker before marking them as non-compacting 
during anti-compaction (CASSANDRA-11548)
 +2.2.6
 + * Allow only DISTINCT queries with partition keys restrictions 
(CASSANDRA-11339)
 + * CqlConfigHelper no longer requires both a keystore and truststore to work 
(CASSANDRA-11532)
 + * Make deprecated repair methods backward-compatible with previous 
notification service (CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
 + * DatabaseDescriptor should log stacktrace in case of Eception during seed 
provider creation (CASSANDRA-11312)
 + * Use canonical path for directory in SSTable descriptor (CASSANDRA-10587)
 + * Add cassandra-stress keystore option (CASSANDRA-9325)
 + * Fix out-of-space error treatment in memtable flushing (CASSANDRA-11448).
 + * Dont mark sstables as repairing with sub range repairs (CASSANDRA-11451)
 + * Fix use of NullUpdater for 2i during compaction (CASSANDRA-11450)
 + * Notify when sstables change after cancelling compaction (CASSANDRA-11373)
 + * cqlsh: COPY FROM should check that explicit column names are valid 
(CASSANDRA-11333)
 + * Add -Dcassandra.start_gossip 

[03/10] cassandra git commit: Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone patch by jbellis; reviewed by marcuse for CASSANDRA-11834

[jbellis]

Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
expired tombstone
patch by jbellis; reviewed by marcuse for CASSANDRA-11834


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00f25401
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00f25401
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00f25401

Branch: refs/heads/cassandra-3.0
Commit: 00f25401c9a320c0c2ac94fbb1014f49597026fd
Parents: 9359af2
Author: Jonathan Ellis 
Authored: Wed May 18 12:24:23 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:25:45 2016 -0500

--
 CHANGES.txt  | 11 ---
 .../cassandra/db/compaction/LazilyCompactedRow.java  |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 68f8dd9..3972248 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,13 +1,18 @@
 2.1.15
- * Fix paging on DISTINCT queries repeats result when first row in partition 
changes (CASSANDRA-11679)
+ * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+   expired tombstone (CASSANDRA-11834)
+ * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
+   (CASSANDRA-11679)
  * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
  * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
  * Make cython optional in pylib/setup.py (CASSANDRA-11630)
- * Change order of directory searching for cassandra.in.sh to favor local one 
(CASSANDRA-11628)
+ * Change order of directory searching for cassandra.in.sh to favor local one 
+   (CASSANDRA-11628)
  * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
  * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
  * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
- * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+ * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+   sstables (CASSANDRA-9935)
 
 
 2.1.14

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java 
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index 941557b..4ca9829 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -271,7 +271,7 @@ public class LazilyCompactedRow extends AbstractCompactedRow
 RangeTombstone t = tombstone;
 tombstone = null;
 
-if (t.timestamp() < getMaxPurgeableTimestamp() && 
t.data.isGcAble(controller.gcBefore))
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() < 
getMaxPurgeableTimestamp())
 {
 indexBuilder.tombstoneTracker().update(t, true);
 return null;

[02/10] cassandra git commit: Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone patch by jbellis; reviewed by marcuse for CASSANDRA-11834

[jbellis]

Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
expired tombstone
patch by jbellis; reviewed by marcuse for CASSANDRA-11834


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00f25401
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00f25401
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00f25401

Branch: refs/heads/cassandra-2.2
Commit: 00f25401c9a320c0c2ac94fbb1014f49597026fd
Parents: 9359af2
Author: Jonathan Ellis 
Authored: Wed May 18 12:24:23 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:25:45 2016 -0500

--
 CHANGES.txt  | 11 ---
 .../cassandra/db/compaction/LazilyCompactedRow.java  |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 68f8dd9..3972248 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,13 +1,18 @@
 2.1.15
- * Fix paging on DISTINCT queries repeats result when first row in partition 
changes (CASSANDRA-11679)
+ * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+   expired tombstone (CASSANDRA-11834)
+ * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
+   (CASSANDRA-11679)
  * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
  * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
  * Make cython optional in pylib/setup.py (CASSANDRA-11630)
- * Change order of directory searching for cassandra.in.sh to favor local one 
(CASSANDRA-11628)
+ * Change order of directory searching for cassandra.in.sh to favor local one 
+   (CASSANDRA-11628)
  * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
  * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
  * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
- * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+ * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+   sstables (CASSANDRA-9935)
 
 
 2.1.14

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java 
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index 941557b..4ca9829 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -271,7 +271,7 @@ public class LazilyCompactedRow extends AbstractCompactedRow
 RangeTombstone t = tombstone;
 tombstone = null;
 
-if (t.timestamp() < getMaxPurgeableTimestamp() && 
t.data.isGcAble(controller.gcBefore))
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() < 
getMaxPurgeableTimestamp())
 {
 indexBuilder.tombstoneTracker().update(t, true);
 return null;

[06/10] cassandra git commit: merge from 2.1

[jbellis]

merge from 2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be653932
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be653932
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be653932

Branch: refs/heads/cassandra-2.2
Commit: be653932d883134ec6ebecd5b9ad1dbb64bda64a
Parents: 89344f1 00f2540
Author: Jonathan Ellis 
Authored: Wed May 18 12:28:45 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:30:14 2016 -0500

--
 CHANGES.txt | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/be653932/CHANGES.txt
--
diff --cc CHANGES.txt
index befb520,3972248..103d057
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,8 +1,24 @@@
-  * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
- 
 -2.1.15
 +2.2.7
++ * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+expired tombstone (CASSANDRA-11834)
 - * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
 -   (CASSANDRA-11679)
   * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
   * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
   * Make cython optional in pylib/setup.py (CASSANDRA-11630)
@@@ -25,62 -11,21 +27,63 @@@
   * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
   * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
   * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
 + * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 +   processes (CASSANDRA-11505)
-  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+  * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+sstables (CASSANDRA-9935)
  
  
 -2.1.14
 - * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 -   going (CASSANDRA-10979)
 - * (cqlsh) Fix potential COPY deadlock when parent process is terminating 
child
 -   processes (CASSANDRA-11505)
 - * Replace sstables on DataTracker before marking them as non-compacting 
during anti-compaction (CASSANDRA-11548)
 +2.2.6
 + * Allow only DISTINCT queries with partition keys restrictions 
(CASSANDRA-11339)
 + * CqlConfigHelper no longer requires both a keystore and truststore to work 
(CASSANDRA-11532)
 + * Make deprecated repair methods backward-compatible with previous 
notification service (CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
 + * DatabaseDescriptor should log stacktrace in case of Eception during seed 
provider creation (CASSANDRA-11312)
 + * Use canonical path for directory in SSTable descriptor (CASSANDRA-10587)
 + * Add cassandra-stress keystore option (CASSANDRA-9325)
 + * Fix out-of-space error treatment in memtable flushing (CASSANDRA-11448).
 + * Dont mark sstables as repairing with sub range repairs (CASSANDRA-11451)
 + * Fix use of NullUpdater for 2i during compaction (CASSANDRA-11450)
 + * Notify when sstables change after cancelling compaction (CASSANDRA-11373)
 + * cqlsh: COPY FROM should check that explicit column names are valid 
(CASSANDRA-11333)
 + * Add -Dcassandra.start_gossip 

[04/10] cassandra git commit: Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone patch by jbellis; reviewed by marcuse for CASSANDRA-11834

[jbellis]

Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
expired tombstone
patch by jbellis; reviewed by marcuse for CASSANDRA-11834


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00f25401
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00f25401
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00f25401

Branch: refs/heads/trunk
Commit: 00f25401c9a320c0c2ac94fbb1014f49597026fd
Parents: 9359af2
Author: Jonathan Ellis 
Authored: Wed May 18 12:24:23 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:25:45 2016 -0500

--
 CHANGES.txt  | 11 ---
 .../cassandra/db/compaction/LazilyCompactedRow.java  |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 68f8dd9..3972248 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,13 +1,18 @@
 2.1.15
- * Fix paging on DISTINCT queries repeats result when first row in partition 
changes (CASSANDRA-11679)
+ * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+   expired tombstone (CASSANDRA-11834)
+ * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
+   (CASSANDRA-11679)
  * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
  * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
  * Make cython optional in pylib/setup.py (CASSANDRA-11630)
- * Change order of directory searching for cassandra.in.sh to favor local one 
(CASSANDRA-11628)
+ * Change order of directory searching for cassandra.in.sh to favor local one 
+   (CASSANDRA-11628)
  * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
  * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
  * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
- * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+ * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+   sstables (CASSANDRA-9935)
 
 
 2.1.14

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java 
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index 941557b..4ca9829 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -271,7 +271,7 @@ public class LazilyCompactedRow extends AbstractCompactedRow
 RangeTombstone t = tombstone;
 tombstone = null;
 
-if (t.timestamp() < getMaxPurgeableTimestamp() && 
t.data.isGcAble(controller.gcBefore))
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() < 
getMaxPurgeableTimestamp())
 {
 indexBuilder.tombstoneTracker().update(t, true);
 return null;

[01/10] cassandra git commit: Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone patch by jbellis; reviewed by marcuse for CASSANDRA-11834

[jbellis]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 9359af2ea -> 00f25401c
  refs/heads/cassandra-2.2 89344f182 -> be653932d
  refs/heads/cassandra-3.0 f73b48502 -> 958183fad
  refs/heads/trunk 050516c53 -> 0f483d8ff


Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
expired tombstone
patch by jbellis; reviewed by marcuse for CASSANDRA-11834


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00f25401
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00f25401
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00f25401

Branch: refs/heads/cassandra-2.1
Commit: 00f25401c9a320c0c2ac94fbb1014f49597026fd
Parents: 9359af2
Author: Jonathan Ellis 
Authored: Wed May 18 12:24:23 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:25:45 2016 -0500

--
 CHANGES.txt  | 11 ---
 .../cassandra/db/compaction/LazilyCompactedRow.java  |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 68f8dd9..3972248 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,13 +1,18 @@
 2.1.15
- * Fix paging on DISTINCT queries repeats result when first row in partition 
changes (CASSANDRA-11679)
+ * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
+   expired tombstone (CASSANDRA-11834)
+ * Fix paging on DISTINCT queries repeats result when first row in partition 
changes 
+   (CASSANDRA-11679)
  * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
  * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
  * Make cython optional in pylib/setup.py (CASSANDRA-11630)
- * Change order of directory searching for cassandra.in.sh to favor local one 
(CASSANDRA-11628)
+ * Change order of directory searching for cassandra.in.sh to favor local one 
+   (CASSANDRA-11628)
  * cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values 
(CASSANDRA-11633)
  * clqsh: COPY FROM throws TypeError with Cython extensions enabled 
(CASSANDRA-11574)
  * cqlsh: COPY FROM ignores NULL values in conversion (CASSANDRA-11549)
- * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned sstables (CASSANDRA-9935)
+ * Validate levels when building LeveledScanner to avoid overlaps with 
orphaned 
+   sstables (CASSANDRA-9935)
 
 
 2.1.14

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00f25401/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java 
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index 941557b..4ca9829 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -271,7 +271,7 @@ public class LazilyCompactedRow extends AbstractCompactedRow
 RangeTombstone t = tombstone;
 tombstone = null;
 
-if (t.timestamp() < getMaxPurgeableTimestamp() && 
t.data.isGcAble(controller.gcBefore))
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() < 
getMaxPurgeableTimestamp())
 {
 indexBuilder.tombstoneTracker().update(t, true);
 return null;

[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jbellis]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f483d8f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f483d8f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f483d8f

Branch: refs/heads/trunk
Commit: 0f483d8ff34a14a2030d6481669988d33d4de2c0
Parents: 050516c 958183f
Author: Jonathan Ellis 
Authored: Wed May 18 12:31:02 2016 -0500
Committer: Jonathan Ellis 
Committed: Wed May 18 12:31:02 2016 -0500

--

--

[jira] [Updated] (CASSANDRA-11679) Cassandra Driver returns different number of results depending on fetchsize

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11679:
---
Fix Version/s: (was: 2.1.x)
   2.1.15

> Cassandra Driver returns different number of results depending on fetchsize
> ---
>
> Key: CASSANDRA-11679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11679
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Varun Barala
>Assignee: Benjamin Lerer
> Fix For: 2.1.15
>
> Attachments: 11679-2.1.txt
>
>
> I'm trying to fetch all distinct keys from a CF using cassandra-driver 
> (2.1.7.1) and I observed some strange behavior :-
> The total distinct rows are 498 so If I perform a query get All distinctKeys 
> It returns 503 instead of 498(five keys twice).
> But If I define the fetch size in select statement more than 498 then it 
> returns exact 498 rows. 
> And If I execute same statement on Dev-center it returns 498 rows (because 
> the default fetch size is 5000). In `cqlsh` it returns 503 rows (because 
> cqlsh uses fetch size=100).
> Some Additional and useful information :- 
> ---
> Cassandra-2.1.13  (C)* version
> Consistency level: ONE 
> local machine(ubuntu 14.04)
> Table Schema:-
> --
> {code:xml}
> CREATE TABLE sample (
>  pk1 text,
>  pk2 text,
> row_id uuid,
> value blob,
> PRIMARY KEY (( pk1,  pk2))
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> query :-
> 
> {code:xml}
> SELECT DISTINCT  pk2, pk1 FROM sample LIMIT 2147483647;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-11834:

Reviewer: Marcus Eriksson

> Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
> expired tombstone
> ---
>
> Key: CASSANDRA-11834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 2.1.15
>
> Attachments: 11834.txt
>
>
> In LCR's get reduced, we currently do this:
> {code}
> if (t.timestamp() < getMaxPurgeableTimestamp() && 
> t.data.isGcAble(controller.gcBefore))
> {code}
> Should call the expensive getMaxPurgeableTimestamp only after we have called 
> the cheap isGcAble.
> Marking this as a bug since it can cause pathological performance problems 
> (CASSANDRA-11831).
> Have verified that this is not a problem in 3.0 (CompactionIterator does the 
> check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289353#comment-15289353
 ] 

Marcus Eriksson commented on CASSANDRA-11834:
-

+1

> Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
> expired tombstone
> ---
>
> Key: CASSANDRA-11834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 2.1.15
>
> Attachments: 11834.txt
>
>
> In LCR's get reduced, we currently do this:
> {code}
> if (t.timestamp() < getMaxPurgeableTimestamp() && 
> t.data.isGcAble(controller.gcBefore))
> {code}
> Should call the expensive getMaxPurgeableTimestamp only after we have called 
> the cheap isGcAble.
> Marking this as a bug since it can cause pathological performance problems 
> (CASSANDRA-11831).
> Have verified that this is not a problem in 3.0 (CompactionIterator does the 
> check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11834:
---
Assignee: Jonathan Ellis
  Status: Patch Available  (was: Open)

> Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
> expired tombstone
> ---
>
> Key: CASSANDRA-11834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 2.1.15
>
> Attachments: 11834.txt
>
>
> In LCR's get reduced, we currently do this:
> {code}
> if (t.timestamp() < getMaxPurgeableTimestamp() && 
> t.data.isGcAble(controller.gcBefore))
> {code}
> Should call the expensive getMaxPurgeableTimestamp only after we have called 
> the cheap isGcAble.
> Marking this as a bug since it can cause pathological performance problems 
> (CASSANDRA-11831).
> Have verified that this is not a problem in 3.0 (CompactionIterator does the 
> check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11834:
---
Priority: Minor  (was: Major)

> Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
> expired tombstone
> ---
>
> Key: CASSANDRA-11834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan Ellis
>Priority: Minor
> Fix For: 2.1.15
>
> Attachments: 11834.txt
>
>
> In LCR's get reduced, we currently do this:
> {code}
> if (t.timestamp() < getMaxPurgeableTimestamp() && 
> t.data.isGcAble(controller.gcBefore))
> {code}
> Should call the expensive getMaxPurgeableTimestamp only after we have called 
> the cheap isGcAble.
> Marking this as a bug since it can cause pathological performance problems 
> (CASSANDRA-11831).
> Have verified that this is not a problem in 3.0 (CompactionIterator does the 
> check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11834:
---
Attachment: 11834.txt

Trivial patch attached.

> Don't compute expensive MaxPurgeableTimestamp until we've verified there's an 
> expired tombstone
> ---
>
> Key: CASSANDRA-11834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan Ellis
> Fix For: 2.1.15
>
> Attachments: 11834.txt
>
>
> In LCR's get reduced, we currently do this:
> {code}
> if (t.timestamp() < getMaxPurgeableTimestamp() && 
> t.data.isGcAble(controller.gcBefore))
> {code}
> Should call the expensive getMaxPurgeableTimestamp only after we have called 
> the cheap isGcAble.
> Marking this as a bug since it can cause pathological performance problems 
> (CASSANDRA-11831).
> Have verified that this is not a problem in 3.0 (CompactionIterator does the 
> check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11834) Don't compute expensive MaxPurgeableTimestamp until we've verified there's an expired tombstone

[Jonathan Ellis (JIRA)]

Jonathan Ellis created CASSANDRA-11834:
--

 Summary: Don't compute expensive MaxPurgeableTimestamp until we've 
verified there's an expired tombstone
 Key: CASSANDRA-11834
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11834
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction
Reporter: Jonathan Ellis
 Fix For: 2.1.15


In LCR's get reduced, we currently do this:

{code}
if (t.timestamp() < getMaxPurgeableTimestamp() && 
t.data.isGcAble(controller.gcBefore))
{code}

Should call the expensive getMaxPurgeableTimestamp only after we have called 
the cheap isGcAble.

Marking this as a bug since it can cause pathological performance problems 
(CASSANDRA-11831).

Have verified that this is not a problem in 3.0 (CompactionIterator does the 
check in the correct order).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)