[jira] [Updated] (CASSANDRA-13792) unable to connect apache-cassandra-3.11.0 after place 4.2.2 JNA jar

[rajasekhar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

rajasekhar updated CASSANDRA-13792:
---
Priority: Critical  (was: Major)

> unable to connect apache-cassandra-3.11.0 after place 4.2.2 JNA jar 
> 
>
> Key: CASSANDRA-13792
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13792
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
> Environment: Linux
>Reporter: rajasekhar
>Priority: Critical
> Fix For: 3.11.0
>
> Attachments: log_cassdb.txt
>
>
> After apply work around below solution, installled the pache-cassandra-3.11 
> but i am not connect server status and cqlsh. please find attached log for 
> more information  and let us know the instatllion completed successfully
> "Jeff Jirsa resolved CASSANDRA-13791.
> 
> Resolution: Duplicate
> This was caused (and resolved) by CASSANDRA-13072 , where JNA was upgraded to 
> {{4.4.0}}. That 4.4.0 JNA library was linked against a different version of 
> glibc ( {{GLIBC_2.14}} ), which causes this error. This is fixed in 
> {{3.11.1}}, but you can work around the issue by downloading the 4.2.2 JNA 
> jar from 
> [here|https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra_raw_00a777ec8ab701b843172e23a6cbdc4d6cf48f8d_lib_jna-2D4.2.2.jar=DwICaQ=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU=zSydaPm_PKES-H2mB46-9X-iWMDIIymvQCfY02eRW-Q=cSmoTlLQJTt32Gb257uuaz6_WHE6UeXOV5XjO7GTcxk=yieYJ2V_ND3saGBjGH3O-DEwQdfkA44B9yBqtjIU8bE=
>  ] and placing it into the classpath ({{lib/}}), removing {{jna-4.4.0.jar}} 
> in the process.
> > unable to install apache-cassandra-3.11.0 in linux  box
> > ---
> "
> [cassdb@alsc_dev_db bin]$ nodetool status
> -bash: nodetool: command not found
> [cassdb@alsc_dev_db bin]$ pwd
> /u01/Cassandra_home/apache-cassandra-3.11.0/bin
> [cassdb@alsc_dev_db bin]$ sh 
> /u01/Cassandra_home/apache-cassandra-3.11.0/bin/cqlsh
> No appropriate python interpreter found.
> [cassdb@alsc_dev_db bin]$ service cassandra status



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141130#comment-16141130
 ] 

mck edited comment on CASSANDRA-13418 at 8/25/17 4:50 AM:
--

{quote}is this bad new ? 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214/
{quote} looks just like a flakey test to me. 

The CHANGES.txt entry was in the wrong place. If this is a patch against trunk, 
then it's to go under 4.0. But a patch would also be nice for 3.11.
I'll update this in thelastpickle repo, see links below, and you can let me 
know if you agree. The commit message has been updated as well, per practice.

These patches are then:

|| branch || testall || dtest ||
| 
[cassandra-3.11_13418|https://github.com/thelastpickle/cassandra/tree/mck/cassandra-3.11_13418]
   | 
[testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_13418]
 | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/219/]
 |
| [trunk_13418|https://github.com/thelastpickle/cassandra/tree/mck/trunk_13418] 
| 
[testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Ftrunk_13418]
  | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214]
 |



was (Author: michaelsembwever):
{quote}is this bad new ? 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214/
{quote} looks just like a flakey test to me. 

The CHANGES.txt entry was in the wrong place. If this is a patch against trunk, 
then it's to go under 4.0. But a patch would also be nice for 3.11.
I'll update this in thelastpickle repo, see links below, and you can let me 
know if you agree. The commit message has been updated as well, per practice.

These patches are then:

|| branch || testall || dtest ||
| 
[cassandra-3.11_13418|https://github.com/thelastpickle/cassandra/tree/mck/cassandra-3.11_13418]
   | 
[testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_13418]
 | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/219/]
 |
| [trunk_13418|https://github.com/thelastpickle/cassandra/tree/mck/trunk_13418] 
| [testall|https://circleci.com/gh/thelastpickle/cassandra/mck%2Ftrunk_13418]   
| 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214]
 |


> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141130#comment-16141130
 ] 

mck edited comment on CASSANDRA-13418 at 8/25/17 4:45 AM:
--

{quote}is this bad new ? 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214/
{quote} looks just like a flakey test to me. 

The CHANGES.txt entry was in the wrong place. If this is a patch against trunk, 
then it's to go under 4.0. But a patch would also be nice for 3.11.
I'll update this in thelastpickle repo, see links below, and you can let me 
know if you agree. The commit message has been updated as well, per practice.

These patches are then:

|| branch || testall || dtest ||
| 
[cassandra-3.11_13418|https://github.com/thelastpickle/cassandra/tree/mck/cassandra-3.11_13418]
   | 
[testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_13418]
 | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/219/]
 |
| [trunk_13418|https://github.com/thelastpickle/cassandra/tree/mck/trunk_13418] 
| [testall|https://circleci.com/gh/thelastpickle/cassandra/mck%2Ftrunk_13418]   
| 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214]
 |



was (Author: michaelsembwever):
{quote}is this bad new ? 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214/
{quote} looks just like a flakey test to me. 

The CHANGES.txt entry was in the wrong place. If this is a patch against trunk, 
then it's to go under 4.0. But a patch would also be nice for 3.11.
I'll update this in thelastpickle repo, see links below, and you can let me 
know if you agree. The commit message has been updated as well, per practice.

These patches are then:

|| branch || testall || dtest ||
| 
[cassandra-3.11_13418|https://github.com/criteo-forks/cassandra/tree/mck/cassandra-3.11_13418]
| 
[testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_13418]
 | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/219/]
 |
| 
[trunk_13418|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Ftrunk_13418]
  | [testall|https://circleci.com/gh/thelastpickle/cassandra/22]  | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214]
 |


> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141130#comment-16141130
 ] 

mck commented on CASSANDRA-13418:
-

{quote}is this bad new ? 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214/
{quote} looks just like a flakey test to me. 

The CHANGES.txt entry was in the wrong place. If this is a patch against trunk, 
then it's to go under 4.0. But a patch would also be nice for 3.11.
I'll update this in thelastpickle repo, see links below, and you can let me 
know if you agree. The commit message has been updated as well, per practice.

These patches are then:

|| branch || testall || dtest ||
| 
[cassandra-3.11_13418|https://github.com/criteo-forks/cassandra/tree/mck/cassandra-3.11_13418]
| 
[testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_13418]
 | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/219/]
 |
| 
[trunk_13418|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Ftrunk_13418]
  | [testall|https://circleci.com/gh/thelastpickle/cassandra/22]  | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214]
 |


> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13798) Disallow filtering on non-primary-key base column for MV

[ZhaoYang (JIRA)]

ZhaoYang created CASSANDRA-13798:


 Summary: Disallow filtering on non-primary-key base column for MV
 Key: CASSANDRA-13798
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13798
 Project: Cassandra
  Issue Type: Bug
  Components: Materialized Views
Reporter: ZhaoYang


We should probably consider disallow filtering conditions on non-primary-key 
base column for Materialized View which is introduced in CASSANDRA-10368.

The main problem is that the liveness of view row is now depending on multiple 
base columns (multiple filtered non-pk base column + base column used in view 
pk) and this semantic could not be properly support without drastic storage 
format changes. (SEE CASSANDRA-11500, 
[background|https://issues.apache.org/jira/browse/CASSANDRA-11500?focusedCommentId=16119823=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16119823])

We should step back and re-consider the non-primary-key filtering feature 
together with supporting multiple non-PK cols in MV clustering key in 
CASSANDRA-10226.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13702) Error on keyspace create/alter if referencing non-existing DC in cluster

[Murukesh Mohanan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Murukesh Mohanan resolved CASSANDRA-13702.
--
   Resolution: Duplicate
Fix Version/s: 4.0

> Error on keyspace create/alter if referencing non-existing DC in cluster
> 
>
> Key: CASSANDRA-13702
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13702
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL, Distributed Metadata
>Reporter: Johnny Miller
>Priority: Minor
>  Labels: lhf
> Fix For: 4.0
>
>
> It is possible to create/alter a keyspace using NetworkTopologyStrategy and a 
> DC that does not exist. It would be great if this was validated to prevent 
> accidents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13702) Error on keyspace create/alter if referencing non-existing DC in cluster

[Atsuya Yamashita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141054#comment-16141054
 ] 

Atsuya Yamashita commented on CASSANDRA-13702:
--

https://github.com/apache/cassandra/commit/f2c5ad743933498e60e7eef55e8daaa6ce338a03

This issue is resolved because of CASSANDRA-12681.

> Error on keyspace create/alter if referencing non-existing DC in cluster
> 
>
> Key: CASSANDRA-13702
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13702
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL, Distributed Metadata
>Reporter: Johnny Miller
>Priority: Minor
>  Labels: lhf
>
> It is possible to create/alter a keyspace using NetworkTopologyStrategy and a 
> DC that does not exist. It would be great if this was validated to prevent 
> accidents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140977#comment-16140977
 ] 

Jay Zhuang edited comment on CASSANDRA-13785 at 8/25/17 12:23 AM:
--

[~rha] that makes sense. I updated the patch to define the magic number.
Basically, the number is coming from here: 
[IndexSummaryBuilder.java:112|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
We potentially could support more entries if the partition key size is smaller 
than {{32}}, but I think it's still good to increase the 
{{[effectiveMinInterval|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L96]}}
 to reduce the summary size automatically in that case.
Also the "large number of keys" more likely to have larger partition key size 
(like {{32}} or even larger).

| branch | dTest |
| [13785-3.0|https://github.com/cooldoger/cassandra/tree/13785-3.0] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.0.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.0]
 |
| [13785-3.11|https://github.com/cooldoger/cassandra/tree/13785-3.11] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.11.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.11]
 |
| [13785-trunk|https://github.com/cooldoger/cassandra/tree/13785-trunk] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-trunk.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-trunk]
 |



was (Author: jay.zhuang):
[~rha] that makes sense. I updated the patch to define the magic number.
Basically, the number is coming from here: 
[IndexSummaryBuilder.java:112|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
We potentially could support more entries if the partition key size is smaller 
than {{32}}, but I think it's still good to increase the 
{{[effectiveMinInterval|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L96]}}
 to reduce the summary size automatically in that case.
Also the "large number of keys" more likely have larger partition key size 
(like {{32}} or even larger).

| branch | dTest |
| [13785-3.0|https://github.com/cooldoger/cassandra/tree/13785-3.0] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.0.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.0]
 |
| [13785-3.11|https://github.com/cooldoger/cassandra/tree/13785-3.11] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.11.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.11]
 |
| [13785-trunk|https://github.com/cooldoger/cassandra/tree/13785-trunk] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-trunk.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-trunk]
 |


> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
>  Memory@[0..d8123468)] was not released before the reference was garbage 
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
>  Memory@[0..d8123468)] was not released before the reference was garbage 
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
>  Memory@[0..d8123468)] was not released before the reference was garbage 
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the 
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will 
> fail to create the IndexSummary: 
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction 

[jira] [Commented] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140977#comment-16140977
 ] 

Jay Zhuang commented on CASSANDRA-13785:


[~rha] that makes sense. I updated the patch to define the magic number.
Basically, the number is coming from here: 
[IndexSummaryBuilder.java:112|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
We potentially could support more entries if the partition key size is smaller 
than {{32}}, but I think it's still good to increase the 
{{[effectiveMinInterval|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L96]}}
 to reduce the summary size automatically in that case.
Also the "large number of keys" more likely have larger partition key size 
(like {{32}} or even larger).

| branch | dTest |
| [13785-3.0|https://github.com/cooldoger/cassandra/tree/13785-3.0] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.0.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.0]
 |
| [13785-3.11|https://github.com/cooldoger/cassandra/tree/13785-3.11] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.11.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-3.11]
 |
| [13785-trunk|https://github.com/cooldoger/cassandra/tree/13785-trunk] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/13785-trunk.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/13785-trunk]
 |


> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
>  Memory@[0..d8123468)] was not released before the reference was garbage 
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
>  Memory@[0..d8123468)] was not released before the reference was garbage 
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class 
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
>  Memory@[0..d8123468)] was not released before the reference was garbage 
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the 
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will 
> fail to create the IndexSummary: 
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction tried to compact every a few minutes and keeps failing.
> The root cause is while [creating 
> SafeMemoryWriter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
>  with {{> Integer.MAX_VALUE}} space, it returns the tailing 
> {{Integer.MAX_VALUE}} space 
> [SafeMemoryWriter.java:83|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L83],
>  which makes the first 
> [entries.length()|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L173]
>  not 0. So the assert fails here: 
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13797) RepairJob blocks on syncTasks

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13797:

Reviewer: Marcus Eriksson
  Status: Patch Available  (was: Open)

[trunk|https://github.com/bdeggleston/cassandra/tree/13797]
[utest|https://circleci.com/gh/bdeggleston/cassandra/104]
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/218/]

> RepairJob blocks on syncTasks
> -
>
> Key: CASSANDRA-13797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13797
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
> Fix For: 4.0
>
>
> The thread running {{RepairJob}} blocks while it waits for the validations it 
> starts to complete ([see 
> here|https://github.com/bdeggleston/cassandra/blob/9fdec0a82851f5c35cd21d02e8c4da8fc685edb2/src/java/org/apache/cassandra/repair/RepairJob.java#L185]).
>  However, the downstream callbacks (ie: the post-repair cleanup stuff) aren't 
> waiting for {{RepairJob#run}} to return, they're waiting for a result to be 
> set on RepairJob the future, which happens after the sync tasks have 
> completed. This post repair cleanup stuff also immediately shuts down the 
> executor {{RepairJob#run}} is running in. So in noop repair sessions, where 
> there's nothing to stream, I'm seeing the callbacks sometimes fire before 
> {{RepairJob#run}} wakes up, and causing an {{InterruptedException}} is thrown.
> I'm pretty sure this can just be removed, but I'd like a second opinion. This 
> appears to just be a holdover from before repair coordination became async. I 
> thought it might be doing some throttling by blocking, but each repair 
> session gets it's own executor, and validation is  throttled by the fixed 
> size executors doing the actual work of validation, so I don't think we need 
> to keep this around.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13797) RepairJob blocks on syncTasks

[Blake Eggleston (JIRA)]

Blake Eggleston created CASSANDRA-13797:
---

 Summary: RepairJob blocks on syncTasks
 Key: CASSANDRA-13797
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13797
 Project: Cassandra
  Issue Type: Bug
Reporter: Blake Eggleston
Assignee: Blake Eggleston
 Fix For: 4.0


The thread running {{RepairJob}} blocks while it waits for the validations it 
starts to complete ([see 
here|https://github.com/bdeggleston/cassandra/blob/9fdec0a82851f5c35cd21d02e8c4da8fc685edb2/src/java/org/apache/cassandra/repair/RepairJob.java#L185]).
 However, the downstream callbacks (ie: the post-repair cleanup stuff) aren't 
waiting for {{RepairJob#run}} to return, they're waiting for a result to be set 
on RepairJob the future, which happens after the sync tasks have completed. 
This post repair cleanup stuff also immediately shuts down the executor 
{{RepairJob#run}} is running in. So in noop repair sessions, where there's 
nothing to stream, I'm seeing the callbacks sometimes fire before 
{{RepairJob#run}} wakes up, and causing an {{InterruptedException}} is thrown.

I'm pretty sure this can just be removed, but I'd like a second opinion. This 
appears to just be a holdover from before repair coordination became async. I 
thought it might be doing some throttling by blocking, but each repair session 
gets it's own executor, and validation is  throttled by the fixed size 
executors doing the actual work of validation, so I don't think we need to keep 
this around.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13789) Reduce memory copies and object creations when acting on ByteBufs

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13789:

Fix Version/s: 4.0

> Reduce memory copies and object creations when acting on  ByteBufs
> --
>
> Key: CASSANDRA-13789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13789
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Fix For: 4.0
>
> Attachments: 
> 0001-CBUtil.sizeOfLongString-encodes-String-to-byte-to-ca.patch, 
> 0001-Reduce-memory-copies-and-object-creations-when-actin.patch
>
>
> There are multiple "low-hanging-fruits" when it comes to reduce memory copies 
> and object allocations when acting on ByteBufs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13789) Reduce memory copies and object creations when acting on ByteBufs

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13789:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

+1. committed to 4.0 as sha {{6bea7de50418a7a7b9e2b8dcf2446701d9cc5b69}}

Thanks!

> Reduce memory copies and object creations when acting on  ByteBufs
> --
>
> Key: CASSANDRA-13789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13789
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Attachments: 
> 0001-CBUtil.sizeOfLongString-encodes-String-to-byte-to-ca.patch, 
> 0001-Reduce-memory-copies-and-object-creations-when-actin.patch
>
>
> There are multiple "low-hanging-fruits" when it comes to reduce memory copies 
> and object allocations when acting on ByteBufs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Reduce memory copies and object creations when acting on ByteBufs

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk a5f70ad4b -> 6bea7de50


Reduce memory copies and object creations when acting on ByteBufs

patch by Norman Maurer; reviewed by jasobrown for CASSANDRA-13789


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6bea7de5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6bea7de5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6bea7de5

Branch: refs/heads/trunk
Commit: 6bea7de50418a7a7b9e2b8dcf2446701d9cc5b69
Parents: a5f70ad
Author: Norman Maurer 
Authored: Tue Aug 22 14:15:16 2017 +0200
Committer: Jason Brown 
Committed: Thu Aug 24 14:57:37 2017 -0700

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/cql3/QueryOptions.java |  2 +-
 .../org/apache/cassandra/cql3/ResultSet.java|  2 +-
 .../org/apache/cassandra/transport/CBUtil.java  | 56 +---
 4 files changed, 41 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6bea7de5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6b3ea2b..d1ef3c9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Reduce memory copies and object creations when acting on ByteBufs 
(CASSANDRA-13789)
  * simplify mx4j configuration (Cassandra-13578)
  * Fix trigger example on 4.0 (CASSANDRA-13796)
  * force minumum timeout value (CASSANDRA-9375)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6bea7de5/src/java/org/apache/cassandra/cql3/QueryOptions.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryOptions.java 
b/src/java/org/apache/cassandra/cql3/QueryOptions.java
index ac80729..e4d443d 100644
--- a/src/java/org/apache/cassandra/cql3/QueryOptions.java
+++ b/src/java/org/apache/cassandra/cql3/QueryOptions.java
@@ -429,7 +429,7 @@ public abstract class QueryOptions
 if (!flags.isEmpty())
 {
 int pageSize = flags.contains(Flag.PAGE_SIZE) ? body.readInt() 
: -1;
-PagingState pagingState = flags.contains(Flag.PAGING_STATE) ? 
PagingState.deserialize(CBUtil.readValue(body), version) : null;
+PagingState pagingState = flags.contains(Flag.PAGING_STATE) ? 
PagingState.deserialize(CBUtil.readValueNoCopy(body), version) : null;
 ConsistencyLevel serialConsistency = 
flags.contains(Flag.SERIAL_CONSISTENCY) ? CBUtil.readConsistencyLevel(body) : 
ConsistencyLevel.SERIAL;
 long timestamp = Long.MIN_VALUE;
 if (flags.contains(Flag.TIMESTAMP))

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6bea7de5/src/java/org/apache/cassandra/cql3/ResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ResultSet.java 
b/src/java/org/apache/cassandra/cql3/ResultSet.java
index 5ed4eee..2bb9997 100644
--- a/src/java/org/apache/cassandra/cql3/ResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/ResultSet.java
@@ -310,7 +310,7 @@ public class ResultSet
 
 PagingState state = null;
 if (flags.contains(Flag.HAS_MORE_PAGES))
-state = PagingState.deserialize(CBUtil.readValue(body), 
version);
+state = 
PagingState.deserialize(CBUtil.readValueNoCopy(body), version);
 
 if (flags.contains(Flag.NO_METADATA))
 return new ResultMetadata(flags, null, columnCount, state);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6bea7de5/src/java/org/apache/cassandra/transport/CBUtil.java
--
diff --git a/src/java/org/apache/cassandra/transport/CBUtil.java 
b/src/java/org/apache/cassandra/transport/CBUtil.java
index 824ad26..d5d733c 100644
--- a/src/java/org/apache/cassandra/transport/CBUtil.java
+++ b/src/java/org/apache/cassandra/transport/CBUtil.java
@@ -58,6 +58,7 @@ public abstract class CBUtil
 {
 public static final boolean USE_HEAP_ALLOCATOR = 
Boolean.getBoolean(Config.PROPERTY_PREFIX + "netty_use_heap_allocator");
 public static final ByteBufAllocator allocator = USE_HEAP_ALLOCATOR ? new 
UnpooledByteBufAllocator(false) : new PooledByteBufAllocator(true);
+private static final int UUID_SIZE = 16;
 
 private final static FastThreadLocal TL_UTF8_DECODER = new 
FastThreadLocal()
 {
@@ -139,9 +140,9 @@ public abstract class CBUtil
 public static void writeString(String str, ByteBuf cb)
 {
 int writerIndex = cb.writerIndex();
-cb.writeShort(0);
-int lengthBytes = ByteBufUtil.writeUtf8(cb, str);
-

[jira] [Commented] (CASSANDRA-13772) Add a skip read validation flag to cassandra-stress

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140771#comment-16140771
 ] 

Jason Brown commented on CASSANDRA-13772:
-

Crap - I forgot to commit the files on trunk, so the merge commit (to trunk, 
from 3.11) is empty. I've added those files and committed as sha 
{{a5f70ad4b84fa301c6e46b68549b4326f09fd9cf}}. Sorry for any confusion.

> Add a skip read validation flag to cassandra-stress
> ---
>
> Key: CASSANDRA-13772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13772
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Noam Hasson
>Assignee: Noam Hasson
>Priority: Minor
> Fix For: 3.11.1, 4.0
>
> Attachments: 13772-trunk.txt
>
>
> When running cassandra-stress with read operations, you must make sure all 
> the data was populated beforehand or else you will get the following errors:
> java.io.IOException: Operation x0 on key(s) [4d31314e32314b395030]: Data 
> returned was not validated
>   at org.apache.cassandra.stress.Operation.error(Operation.java:127)
>   at 
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:91)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:245)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:453)
> java.lang.RuntimeException: Failed to execute warmup
>   at 
> org.apache.cassandra.stress.StressAction.warmup(StressAction.java:117)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:62)
>   at org.apache.cassandra.stress.Stress.run(Stress.java:143)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Even if you use the "-errors ignore" flag you'll get a lot of the following 
> messages which will both slow the stress and prevent it from displaying the 
> metrics:
> Operation x0 on key(s) [4b3539393831374e3431]: Data returned was not validated
> Operation x0 on key(s) [4f4b3936363233375030]: Data returned was not validated
> Operation x0 on key(s) [4d304c4b32384f4b3031]: Data returned was not validated
> What I propose is to add a flag to skip the read validation, such as:
> -errors skip-read-validations
> This way when needed you can run a mixed workload and ignore validation of 
> unpopulated data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: really commit CASSANDRA-13722 to trunk

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk b19687691 -> a5f70ad4b


really commit CASSANDRA-13722 to trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a5f70ad4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a5f70ad4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a5f70ad4

Branch: refs/heads/trunk
Commit: a5f70ad4b84fa301c6e46b68549b4326f09fd9cf
Parents: b196876
Author: Jason Brown 
Authored: Thu Aug 24 14:47:31 2017 -0700
Committer: Jason Brown 
Committed: Thu Aug 24 14:47:31 2017 -0700

--
 CHANGES.txt  |  1 +
 .../stress/operations/predefined/CqlOperation.java   | 11 +++
 .../apache/cassandra/stress/settings/SettingsErrors.java |  6 --
 3 files changed, 12 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a5f70ad4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cdcfe25..6b3ea2b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -125,6 +125,7 @@
 
 
 3.11.1
+ * Add a skip read validation flag to cassandra-stress (CASSANDRA-13772)
  * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
  * Better bootstrap failure message when blocked by (potential) range movement 
(CASSANDRA-13744)
  * "ignore" option is ignored in sstableloader (CASSANDRA-13721)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a5f70ad4/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
 
b/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
index 7fcc63d..c524107 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
@@ -186,11 +186,14 @@ public abstract class CqlOperation extends 
PredefinedOperation
 
 public boolean validate(ByteBuffer[][] result)
 {
-if (result.length != expect.size())
-return false;
-for (int i = 0 ; i < result.length ; i++)
-if (expect.get(i) != null && 
!expect.get(i).equals(Arrays.asList(result[i])))
+if (!settings.errors.skipReadValidation)
+{
+if (result.length != expect.size())
 return false;
+for (int i = 0; i < result.length; i++)
+if (expect.get(i) != null && 
!expect.get(i).equals(Arrays.asList(result[i])))
+return false;
+}
 return true;
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a5f70ad4/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java 
b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
index aae4dd9..41a3dbd 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
@@ -33,11 +33,13 @@ public class SettingsErrors implements Serializable
 
 public final boolean ignore;
 public final int tries;
+public final boolean skipReadValidation;
 
 public SettingsErrors(Options options)
 {
 ignore = options.ignore.setByUser();
 this.tries = Math.max(1, Integer.parseInt(options.retries.value()) + 
1);
+skipReadValidation = options.skipReadValidation.setByUser();
 }
 
 // Option Declarations
@@ -46,11 +48,11 @@ public class SettingsErrors implements Serializable
 {
 final OptionSimple retries = new OptionSimple("retries=", "[0-9]+", 
"9", "Number of tries to perform for each operation before failing", false);
 final OptionSimple ignore = new OptionSimple("ignore", "", null, "Do 
not fail on errors", false);
-
+final OptionSimple skipReadValidation = new 
OptionSimple("skip-read-validation", "", null, "Skip read validation and 
message output", false);
 @Override
 public List options()
 {
-return Arrays.asList(retries, ignore);
+return Arrays.asList(retries, ignore, skipReadValidation);
 }
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For 

[jira] [Updated] (CASSANDRA-13772) Add a skip read validation flag to cassandra-stress

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13772:

   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   4.0
   3.11.1
   Status: Resolved  (was: Patch Available)

committed as sha {{6b7d73a49695c0ceb78bc7a003ace606a806c13a}}

Thanks!

> Add a skip read validation flag to cassandra-stress
> ---
>
> Key: CASSANDRA-13772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13772
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Noam Hasson
>Assignee: Noam Hasson
>Priority: Minor
> Fix For: 3.11.1, 4.0
>
> Attachments: 13772-trunk.txt
>
>
> When running cassandra-stress with read operations, you must make sure all 
> the data was populated beforehand or else you will get the following errors:
> java.io.IOException: Operation x0 on key(s) [4d31314e32314b395030]: Data 
> returned was not validated
>   at org.apache.cassandra.stress.Operation.error(Operation.java:127)
>   at 
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:91)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:245)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:453)
> java.lang.RuntimeException: Failed to execute warmup
>   at 
> org.apache.cassandra.stress.StressAction.warmup(StressAction.java:117)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:62)
>   at org.apache.cassandra.stress.Stress.run(Stress.java:143)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Even if you use the "-errors ignore" flag you'll get a lot of the following 
> messages which will both slow the stress and prevent it from displaying the 
> metrics:
> Operation x0 on key(s) [4b3539393831374e3431]: Data returned was not validated
> Operation x0 on key(s) [4f4b3936363233375030]: Data returned was not validated
> Operation x0 on key(s) [4d304c4b32384f4b3031]: Data returned was not validated
> What I propose is to add a flag to skip the read validation, such as:
> -errors skip-read-validations
> This way when needed you can run a mixed workload and ignore validation of 
> unpopulated data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jasobrown]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b1968769
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b1968769
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b1968769

Branch: refs/heads/trunk
Commit: b196876914c41de805c15efbb6a3155565724514
Parents: e9da857 6b7d73a
Author: Jason Brown 
Authored: Thu Aug 24 10:57:58 2017 -0700
Committer: Jason Brown 
Committed: Thu Aug 24 10:57:58 2017 -0700

--

--



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/3] cassandra git commit: Add support for skip read validation flag for cassandra-stress

[jasobrown]

Add support for skip read validation flag for cassandra-stress

patch by Noam Hasson; reviewed by jasobrown for CASSANDRA-13772


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b7d73a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b7d73a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b7d73a4

Branch: refs/heads/trunk
Commit: 6b7d73a49695c0ceb78bc7a003ace606a806c13a
Parents: 8d98a99
Author: Noam Hasson 
Authored: Thu Aug 17 11:53:45 2017 +0300
Committer: Jason Brown 
Committed: Thu Aug 24 10:50:20 2017 -0700

--
 .../stress/operations/predefined/CqlOperation.java   | 11 +++
 .../apache/cassandra/stress/settings/SettingsErrors.java |  6 --
 2 files changed, 11 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b7d73a4/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
 
b/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
index 647ba87..807bb49 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
@@ -196,11 +196,14 @@ public abstract class CqlOperation extends 
PredefinedOperation
 
 public boolean validate(ByteBuffer[][] result)
 {
-if (result.length != expect.size())
-return false;
-for (int i = 0 ; i < result.length ; i++)
-if (expect.get(i) != null && 
!expect.get(i).equals(Arrays.asList(result[i])))
+if (!settings.errors.skipReadValidation)
+{
+if (result.length != expect.size())
 return false;
+for (int i = 0; i < result.length; i++)
+if (expect.get(i) != null && 
!expect.get(i).equals(Arrays.asList(result[i])))
+return false;
+}
 return true;
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b7d73a4/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java 
b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
index aae4dd9..41a3dbd 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
@@ -33,11 +33,13 @@ public class SettingsErrors implements Serializable
 
 public final boolean ignore;
 public final int tries;
+public final boolean skipReadValidation;
 
 public SettingsErrors(Options options)
 {
 ignore = options.ignore.setByUser();
 this.tries = Math.max(1, Integer.parseInt(options.retries.value()) + 
1);
+skipReadValidation = options.skipReadValidation.setByUser();
 }
 
 // Option Declarations
@@ -46,11 +48,11 @@ public class SettingsErrors implements Serializable
 {
 final OptionSimple retries = new OptionSimple("retries=", "[0-9]+", 
"9", "Number of tries to perform for each operation before failing", false);
 final OptionSimple ignore = new OptionSimple("ignore", "", null, "Do 
not fail on errors", false);
-
+final OptionSimple skipReadValidation = new 
OptionSimple("skip-read-validation", "", null, "Skip read validation and 
message output", false);
 @Override
 public List options()
 {
-return Arrays.asList(retries, ignore);
+return Arrays.asList(retries, ignore, skipReadValidation);
 }
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/3] cassandra git commit: Add support for skip read validation flag for cassandra-stress

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 8d98a9923 -> 6b7d73a49
  refs/heads/trunk e9da85723 -> b19687691


Add support for skip read validation flag for cassandra-stress

patch by Noam Hasson; reviewed by jasobrown for CASSANDRA-13772


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b7d73a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b7d73a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b7d73a4

Branch: refs/heads/cassandra-3.11
Commit: 6b7d73a49695c0ceb78bc7a003ace606a806c13a
Parents: 8d98a99
Author: Noam Hasson 
Authored: Thu Aug 17 11:53:45 2017 +0300
Committer: Jason Brown 
Committed: Thu Aug 24 10:50:20 2017 -0700

--
 .../stress/operations/predefined/CqlOperation.java   | 11 +++
 .../apache/cassandra/stress/settings/SettingsErrors.java |  6 --
 2 files changed, 11 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b7d73a4/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
 
b/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
index 647ba87..807bb49 100644
--- 
a/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
+++ 
b/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java
@@ -196,11 +196,14 @@ public abstract class CqlOperation extends 
PredefinedOperation
 
 public boolean validate(ByteBuffer[][] result)
 {
-if (result.length != expect.size())
-return false;
-for (int i = 0 ; i < result.length ; i++)
-if (expect.get(i) != null && 
!expect.get(i).equals(Arrays.asList(result[i])))
+if (!settings.errors.skipReadValidation)
+{
+if (result.length != expect.size())
 return false;
+for (int i = 0; i < result.length; i++)
+if (expect.get(i) != null && 
!expect.get(i).equals(Arrays.asList(result[i])))
+return false;
+}
 return true;
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b7d73a4/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java 
b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
index aae4dd9..41a3dbd 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsErrors.java
@@ -33,11 +33,13 @@ public class SettingsErrors implements Serializable
 
 public final boolean ignore;
 public final int tries;
+public final boolean skipReadValidation;
 
 public SettingsErrors(Options options)
 {
 ignore = options.ignore.setByUser();
 this.tries = Math.max(1, Integer.parseInt(options.retries.value()) + 
1);
+skipReadValidation = options.skipReadValidation.setByUser();
 }
 
 // Option Declarations
@@ -46,11 +48,11 @@ public class SettingsErrors implements Serializable
 {
 final OptionSimple retries = new OptionSimple("retries=", "[0-9]+", 
"9", "Number of tries to perform for each operation before failing", false);
 final OptionSimple ignore = new OptionSimple("ignore", "", null, "Do 
not fail on errors", false);
-
+final OptionSimple skipReadValidation = new 
OptionSimple("skip-read-validation", "", null, "Skip read validation and 
message output", false);
 @Override
 public List options()
 {
-return Arrays.asList(retries, ignore);
+return Arrays.asList(retries, ignore, skipReadValidation);
 }
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13786) Validation compactions can cause orphan sstable warnings

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140617#comment-16140617
 ] 

Blake Eggleston commented on CASSANDRA-13786:
-

[~kohlisankalp] and I were talking about this, and he mentioned that even if 
the operator can't do anything to fix, it's still good to know it's happening. 
I coded up a more "correct" solution. [This 
branch|https://github.com/bdeggleston/cassandra/tree/orphan-sstables2] adjusts 
how mutating repairedAt is synchronized, and how getScanners is synchronized. 
So that the scenario causing the warning in this case shouldn't be able to 
happen.

1. for {{getScanners}}, it moves the organizing of sstables into 
unrepaired/pending/repaired into the same read locked block that actually gets 
the scanners
2. adds a {{mutateRepaired}} method to {{CompactionStrategyManager}} that is 
used to mutate the repaired time and move the sstables between compaction 
strategies with a write lock held.

> Validation compactions can cause orphan sstable warnings
> 
>
> Key: CASSANDRA-13786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13786
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> I've seen LevelledCompactionStrategy occasionally logging: 
> {quote} from level 0 is not on corresponding level in the 
> leveled manifest. This is not a problem per se, but may indicate an orphaned 
> sstable due to a failed compaction not cleaned up properly."{quote} warnings 
> from a ValidationExecutor thread.
> What's happening here is that a compaction running concurrently with the 
> validation is promoting (or demoting) sstables as part of an incremental 
> repair, and an sstable has changed hands by the time the validation 
> compaction gets around to getting scanners for it. The sstable 
> isolation/synchronization done by validation compactions is a lot looser than 
> normal compactions, so seeing this happen isn't very surprising. Given that 
> it's harmless, and not unexpected, I think it would be best to not log these 
> during validation compactions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13620) Don't skip corrupt sstables on startup

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13620:
---
Status: Ready to Commit  (was: Patch Available)

> Don't skip corrupt sstables on startup
> --
>
> Key: CASSANDRA-13620
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13620
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: 13620-3.0.png, 13620-3.11.png, 13620-trunk.png
>
>
> If we get an IOException when opening an sstable on startup, we just 
> [skip|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L563-L567]
>  it and continue starting
> we should use the DiskFailurePolicy and never explicitly catch an IOException 
> here



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13620) Don't skip corrupt sstables on startup

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140531#comment-16140531
 ] 

Ariel Weisberg commented on CASSANDRA-13620:


I recognize all of them except ttl_is_respected_on_repair and test_13691.

13691 failed on the 3.0 branch. ttl_is_respected_on_repair failed maybe once in 
the last 30 builds on the 3.0 branch, but not on 3.11 or trunk.

The issue looks like the environment?


{noformat}
Error Message

[Errno 20] Not a directory: '/tmp/dtest-Bx7qYJ/test/node2/bin/cassandra'
 >> begin captured logging << 
dtest: DEBUG: cluster ccm directory: /tmp/dtest-Bx7qYJ
dtest: DEBUG: Done setting configuration options:
{   'initial_token': None,
'num_tokens': '32',
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 1,
'read_request_timeout_in_ms': 1,
'request_timeout_in_ms': 1,
'truncate_request_timeout_in_ms': 1,
'write_request_timeout_in_ms': 1}
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 512.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 512.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 512.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 512.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 512.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 512.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.connection: WARNING: Heartbeat failed for connection 
(140182908452560) to 127.0.0.2
cassandra.cluster: WARNING: Host 127.0.0.2 has been marked down
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 2.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 4.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 8.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, scheduling 
retry in 16.0 seconds: [Errno 111] Tried connecting to [('127.0.0.2', 9042)]. 
Last error: Connection refused
- >> end captured logging << -
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 320, in run
self.setUp()
  File 
"/home/jenkins/jenkins-slave/workspace/Cassandra-devbranch-dtest/cassandra-dtest/ttl_test.py",
 line 349, in setUp
self.cluster.populate(2).start()
  File 
"/home/jenkins/jenkins-slave/workspace/Cassandra-devbranch-dtest/venv/local/lib/python2.7/site-packages/ccmlib/cluster.py",
 line 393, in start
p = node.start(update_pid=False, jvm_args=jvm_args, 
profile_options=profile_options, verbose=verbose, quiet_start=quiet_start, 
allow_root=allow_root)
  File 
"/home/jenkins/jenkins-slave/workspace/Cassandra-devbranch-dtest/venv/local/lib/python2.7/site-packages/ccmlib/node.py",
 line 617, in start
os.chmod(launch_bin, os.stat(launch_bin).st_mode | stat.S_IEXEC)
"[Errno 20] Not a directory: 
'/tmp/dtest-Bx7qYJ/test/node2/bin/cassandra'\n >> begin 
captured logging << \ndtest: DEBUG: cluster ccm directory: 
/tmp/dtest-Bx7qYJ\ndtest: DEBUG: Done setting configuration options:\n{   
'initial_token': None,\n'num_tokens': '32',\n'phi_convict_threshold': 
5,\n'range_request_timeout_in_ms': 1,\n
'read_request_timeout_in_ms': 1,\n'request_timeout_in_ms': 1,\n
'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
1}\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.2, 
scheduling retry in 512.0 seconds: [Errno 111] Tried connecting to 
[('127.0.0.2', 9042)]. Last error: Connection refused\ncassandra.pool: WARNING: 
Error attempting to reconnect to 127.0.0.2, scheduling retry in 512.0 seconds: 
[Errno 111] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection 
refused\ncassandra.pool: WARNING: Error attempting to 

[jira] [Commented] (CASSANDRA-13772) Add a skip read validation flag to cassandra-stress

[Noam Hasson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140523#comment-16140523
 ] 

Noam Hasson commented on CASSANDRA-13772:
-

[~jasobrown] Sorry, I've tested the code was running, probably missed a 
copy-paste when I created the patch.

Thank you!


> Add a skip read validation flag to cassandra-stress
> ---
>
> Key: CASSANDRA-13772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13772
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Noam Hasson
>Assignee: Noam Hasson
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: 13772-trunk.txt
>
>
> When running cassandra-stress with read operations, you must make sure all 
> the data was populated beforehand or else you will get the following errors:
> java.io.IOException: Operation x0 on key(s) [4d31314e32314b395030]: Data 
> returned was not validated
>   at org.apache.cassandra.stress.Operation.error(Operation.java:127)
>   at 
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:91)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:245)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:453)
> java.lang.RuntimeException: Failed to execute warmup
>   at 
> org.apache.cassandra.stress.StressAction.warmup(StressAction.java:117)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:62)
>   at org.apache.cassandra.stress.Stress.run(Stress.java:143)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Even if you use the "-errors ignore" flag you'll get a lot of the following 
> messages which will both slow the stress and prevent it from displaying the 
> metrics:
> Operation x0 on key(s) [4b3539393831374e3431]: Data returned was not validated
> Operation x0 on key(s) [4f4b3936363233375030]: Data returned was not validated
> Operation x0 on key(s) [4d304c4b32384f4b3031]: Data returned was not validated
> What I propose is to add a flag to skip the read validation, such as:
> -errors skip-read-validations
> This way when needed you can run a mixed workload and ignore validation of 
> unpopulated data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables

[Romain GERARD (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140505#comment-16140505
 ] 

Romain GERARD commented on CASSANDRA-13418:
---

is this bad new ? 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/214/


> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9375) force minumum timeout value

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140450#comment-16140450
 ] 

Joshua McKenzie commented on CASSANDRA-9375:


bq. I made the executive decision to commit to trunk (4.0) only.
Happy to +1 that reasoning after the fact.

> force minumum timeout value 
> 
>
> Key: CASSANDRA-9375
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9375
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>Assignee: Varun Barala
>Priority: Trivial
>  Labels: patch
> Fix For: 4.0
>
> Attachments: CASSANDRA-9375_after_review, 
> CASSANDRA-9375_after_review_2.patch, CASSANDRA-9375.patch
>
>
> Granted, this is a nonsensical setting, but the error message makes it tough 
> to discern what's wrong:
> {noformat}
> ERROR 17:13:28,726 Exception encountered during startup
> java.lang.ExceptionInInitializerError
>  at 
> org.apache.cassandra.net.MessagingService.instance(MessagingService.java:310)
>  at 
> org.apache.cassandra.service.StorageService.(StorageService.java:233)
>  at 
> org.apache.cassandra.service.StorageService.(StorageService.java:141)
>  at 
> org.apache.cassandra.locator.DynamicEndpointSnitch.(DynamicEndpointSnitch.java:87)
>  at 
> org.apache.cassandra.locator.DynamicEndpointSnitch.(DynamicEndpointSnitch.java:63)
>  at 
> org.apache.cassandra.config.DatabaseDescriptor.createEndpointSnitch(DatabaseDescriptor.java:518)
>  at 
> org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:350)
>  at 
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:112)
>  at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:213)
>  at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:656)
> Caused by: java.lang.IllegalArgumentException
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:586)
>  at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.scheduleWithFixedDelay(DebuggableScheduledThreadPoolExecutor.java:64)
>  at org.apache.cassandra.utils.ExpiringMap.(ExpiringMap.java:103)
>  at 
> org.apache.cassandra.net.MessagingService.(MessagingService.java:360)
>  at org.apache.cassandra.net.MessagingService.(MessagingService.java:68)
>  at 
> org.apache.cassandra.net.MessagingService$MSHandle.(MessagingService.java:306)
>  ... 11 more
> java.lang.ExceptionInInitializerError
>  at 
> org.apache.cassandra.net.MessagingService.instance(MessagingService.java:310)
>  at 
> org.apache.cassandra.service.StorageService.(StorageService.java:233)
>  at 
> org.apache.cassandra.service.StorageService.(StorageService.java:141)
>  at 
> org.apache.cassandra.locator.DynamicEndpointSnitch.(DynamicEndpointSnitch.java:87)
>  at 
> org.apache.cassandra.locator.DynamicEndpointSnitch.(DynamicEndpointSnitch.java:63)
>  at 
> org.apache.cassandra.config.DatabaseDescriptor.createEndpointSnitch(DatabaseDescriptor.java:518)
>  at 
> org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:350)
>  at 
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:112)
>  at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:213)
>  at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:656)
> Caused by: java.lang.IllegalArgumentException
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:586)
>  at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.scheduleWithFixedDelay(DebuggableScheduledThreadPoolExecutor.java:64)
>  at org.apache.cassandra.utils.ExpiringMap.(ExpiringMap.java:103)
>  at 
> org.apache.cassandra.net.MessagingService.(MessagingService.java:360)
>  at org.apache.cassandra.net.MessagingService.(MessagingService.java:68)
>  at 
> org.apache.cassandra.net.MessagingService$MSHandle.(MessagingService.java:306)
>  ... 11 more
> Exception encountered during startup: null
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13772) Add a skip read validation flag to cassandra-stress

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140404#comment-16140404
 ] 

Jason Brown commented on CASSANDRA-13772:
-

The patch as submitted was largely fine, except for the fact that it didn't 
actually work. [~n...@scylladb.com] forgot just one line in the submission, the 
one that hooks it up in {{SettingsErrors}}:

{code}
-return Arrays.asList(retries, ignore);
+return Arrays.asList(retries, ignore, skipReadValidation);
{code}

Other than that, I'm +1. Will commit shortly. 

> Add a skip read validation flag to cassandra-stress
> ---
>
> Key: CASSANDRA-13772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13772
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Noam Hasson
>Assignee: Noam Hasson
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: 13772-trunk.txt
>
>
> When running cassandra-stress with read operations, you must make sure all 
> the data was populated beforehand or else you will get the following errors:
> java.io.IOException: Operation x0 on key(s) [4d31314e32314b395030]: Data 
> returned was not validated
>   at org.apache.cassandra.stress.Operation.error(Operation.java:127)
>   at 
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:91)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:245)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:453)
> java.lang.RuntimeException: Failed to execute warmup
>   at 
> org.apache.cassandra.stress.StressAction.warmup(StressAction.java:117)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:62)
>   at org.apache.cassandra.stress.Stress.run(Stress.java:143)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Even if you use the "-errors ignore" flag you'll get a lot of the following 
> messages which will both slow the stress and prevent it from displaying the 
> metrics:
> Operation x0 on key(s) [4b3539393831374e3431]: Data returned was not validated
> Operation x0 on key(s) [4f4b3936363233375030]: Data returned was not validated
> Operation x0 on key(s) [4d304c4b32384f4b3031]: Data returned was not validated
> What I propose is to add a flag to skip the read validation, such as:
> -errors skip-read-validations
> This way when needed you can run a mixed workload and ignore validation of 
> unpopulated data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12148) Improve determinism of CDC data availability

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140375#comment-16140375
 ] 

ASF GitHub Bot commented on CASSANDRA-12148:


GitHub user josh-mckenzie opened a pull request:

https://github.com/apache/cassandra-dtest/pull/6

12148 style

See: https://issues.apache.org/jira/browse/CASSANDRA-12148

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/josh-mckenzie/apache-cassandra-dtest 
12148_style

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cassandra-dtest/pull/6.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #6


commit eac28ac1a7e6e60ec5e4c362cbc496cc726819d6
Author: Philip Thompson 
Date:   2017-07-13T08:46:28Z

Push readme update

commit 316902f825fcb5459c9aea9e280f716048c42dbb
Author: Josh McKenzie 
Date:   2016-08-03T17:30:25Z

CASSANDRA-12148 cdc hard-link support




> Improve determinism of CDC data availability
> 
>
> Key: CASSANDRA-12148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12148
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>  Labels: client-impacting, native_protocol
> Fix For: 4.0
>
>
> The latency with which CDC data becomes available has a known limitation due 
> to our reliance on CommitLogSegments being discarded to have the data 
> available in cdc_raw: if a slowly written table co-habitates a 
> CommitLogSegment with CDC data, the CommitLogSegment won't be flushed until 
> we hit either memory pressure on memtables or CommitLog limit pressure. 
> Ultimately, this leaves a non-deterministic element to when data becomes 
> available for CDC consumption unless a consumer parses live CommitLogSegments.
> To work around this limitation and make semi-realtime CDC consumption more 
> friendly to end-users, I propose we extend CDC as follows:
> h6. High level:
> * Consumers parse hard links of active CommitLogSegments in cdc_raw instead 
> of waiting for flush/discard and file move
> * C* stores an offset of the highest seen CDC mutation in a separate idx file 
> per commit log segment in cdc_raw. Clients tail this index file, delta their 
> local last parsed offset on change, and parse the corresponding commit log 
> segment using their last parsed offset as min
> * C* flags that index file with an offset and DONE when the file is flushed 
> so clients know when they can clean up
> h6. Details:
> * On creation of a CommitLogSegment, also hard-link the file in cdc_raw
> * On first write of a CDC-enabled mutation to a segment, we:
> ** Flag it as {{CDCState.CONTAINS}}
> ** Set a long tracking the {{CommitLogPosition}} of the 1st CDC-enabled 
> mutation in the log
> ** Set a long in the CommitLogSegment tracking the offset of the end of the 
> last written CDC mutation in the segment if higher than the previously known 
> highest CDC offset
> * On subsequent writes to the segment, we update the offset of the highest 
> known CDC data
> * On CommitLogSegment fsync, we write a file in cdc_raw as 
> _cdc.idx containing the min offset and end offset fsynced to 
> disk per file
> * On segment discard, if CDCState == {{CDCState.PERMITTED}}, delete both the 
> segment in commitlog and in cdc_raw
> * On segment discard, if CDCState == {{CDCState.CONTAINS}}, delete the 
> segment in commitlog and update the _cdc.idx file w/end offset 
> and a DONE marker
> * On segment replay, store the highest end offset of seen CDC-enabled 
> mutations from a segment and write that to _cdc.idx on 
> completion of segment replay. This should bridge the potential correctness 
> gap of a node writing to a segment and then dying before it can write the 
> _cdc.idx file.
> This should allow clients to skip the beginning of a file to the 1st CDC 
> mutation, track an offset of how far they've parsed, delta against the 
> _cdc.idx file end offset, and use that as a determinant on when to parse new 
> CDC data. Any existing clients written to the initial implementation of CDC 
> need only add the _cdc.idx logic and checking for DONE marker 
> to their code, so the burden on users to update to support this should be 
> quite small for the benefit of having data available as soon as it's fsynced 
> instead of at a non-deterministic time when potentially unrelated tables are 
> flushed.
> Finally, we should look into extending the interface on CommitLogReader to be 
> more friendly for realtime parsing, perhaps supporting taking a 
> CommitLogDescriptor and RandomAccessReader and resuming readSection calls, 
> 

[jira] [Updated] (CASSANDRA-12148) Improve determinism of CDC data availability

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12148:

   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Ready to Commit)

[Committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=e9da85723a8dd40872c4bca087a03b655bd2cacb].
 Will push dtest changes shortly.

> Improve determinism of CDC data availability
> 
>
> Key: CASSANDRA-12148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12148
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>  Labels: client-impacting, native_protocol
> Fix For: 4.0
>
>
> The latency with which CDC data becomes available has a known limitation due 
> to our reliance on CommitLogSegments being discarded to have the data 
> available in cdc_raw: if a slowly written table co-habitates a 
> CommitLogSegment with CDC data, the CommitLogSegment won't be flushed until 
> we hit either memory pressure on memtables or CommitLog limit pressure. 
> Ultimately, this leaves a non-deterministic element to when data becomes 
> available for CDC consumption unless a consumer parses live CommitLogSegments.
> To work around this limitation and make semi-realtime CDC consumption more 
> friendly to end-users, I propose we extend CDC as follows:
> h6. High level:
> * Consumers parse hard links of active CommitLogSegments in cdc_raw instead 
> of waiting for flush/discard and file move
> * C* stores an offset of the highest seen CDC mutation in a separate idx file 
> per commit log segment in cdc_raw. Clients tail this index file, delta their 
> local last parsed offset on change, and parse the corresponding commit log 
> segment using their last parsed offset as min
> * C* flags that index file with an offset and DONE when the file is flushed 
> so clients know when they can clean up
> h6. Details:
> * On creation of a CommitLogSegment, also hard-link the file in cdc_raw
> * On first write of a CDC-enabled mutation to a segment, we:
> ** Flag it as {{CDCState.CONTAINS}}
> ** Set a long tracking the {{CommitLogPosition}} of the 1st CDC-enabled 
> mutation in the log
> ** Set a long in the CommitLogSegment tracking the offset of the end of the 
> last written CDC mutation in the segment if higher than the previously known 
> highest CDC offset
> * On subsequent writes to the segment, we update the offset of the highest 
> known CDC data
> * On CommitLogSegment fsync, we write a file in cdc_raw as 
> _cdc.idx containing the min offset and end offset fsynced to 
> disk per file
> * On segment discard, if CDCState == {{CDCState.PERMITTED}}, delete both the 
> segment in commitlog and in cdc_raw
> * On segment discard, if CDCState == {{CDCState.CONTAINS}}, delete the 
> segment in commitlog and update the _cdc.idx file w/end offset 
> and a DONE marker
> * On segment replay, store the highest end offset of seen CDC-enabled 
> mutations from a segment and write that to _cdc.idx on 
> completion of segment replay. This should bridge the potential correctness 
> gap of a node writing to a segment and then dying before it can write the 
> _cdc.idx file.
> This should allow clients to skip the beginning of a file to the 1st CDC 
> mutation, track an offset of how far they've parsed, delta against the 
> _cdc.idx file end offset, and use that as a determinant on when to parse new 
> CDC data. Any existing clients written to the initial implementation of CDC 
> need only add the _cdc.idx logic and checking for DONE marker 
> to their code, so the burden on users to update to support this should be 
> quite small for the benefit of having data available as soon as it's fsynced 
> instead of at a non-deterministic time when potentially unrelated tables are 
> flushed.
> Finally, we should look into extending the interface on CommitLogReader to be 
> more friendly for realtime parsing, perhaps supporting taking a 
> CommitLogDescriptor and RandomAccessReader and resuming readSection calls, 
> assuming the reader is at the start of a SyncSegment. Would probably also 
> need to rewind to the start of the segment before returning so subsequent 
> calls would respect this contract. This would skip needing to deserialize the 
> descriptor and all completed SyncSegments to get to the root of the desired 
> segment for parsing.
> One alternative we discussed offline - instead of just storing the highest 
> seen CDC offset, we could instead store an offset per CDC mutation 
> (potentially delta encoded) in the idx file to allow clients to seek and only 
> parse the mutations with CDC enabled. My hunch is that the performance delta 
> from doing so wouldn't justify the complexity 

cassandra git commit: Make CDC availability more deterministic via hard-linking

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4838e81a6 -> e9da85723


Make CDC availability more deterministic via hard-linking

Patch by jmckenzie; reviewed by blambov for CASSANDRA-12148


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e9da8572
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e9da8572
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e9da8572

Branch: refs/heads/trunk
Commit: e9da85723a8dd40872c4bca087a03b655bd2cacb
Parents: 4838e81
Author: Josh McKenzie 
Authored: Tue Jul 12 12:53:55 2016 -0400
Committer: Josh McKenzie 
Committed: Thu Aug 24 13:24:00 2017 -0400

--
 CHANGES.txt |   1 +
 NEWS.txt|   6 +-
 doc/source/operating/cdc.rst|  31 +-
 .../cassandra/config/DatabaseDescriptor.java|   4 +
 .../AbstractCommitLogSegmentManager.java|  20 +-
 .../cassandra/db/commitlog/CommitLog.java   |   8 +-
 .../db/commitlog/CommitLogDescriptor.java   |   5 +
 .../db/commitlog/CommitLogPosition.java |   1 +
 .../db/commitlog/CommitLogReadHandler.java  |   2 +-
 .../cassandra/db/commitlog/CommitLogReader.java |  10 +-
 .../db/commitlog/CommitLogReplayer.java |  57 ++-
 .../db/commitlog/CommitLogSegment.java  |  60 +++-
 .../commitlog/CommitLogSegmentManagerCDC.java   |  77 ++--
 .../CommitLogSegmentManagerStandard.java|  14 +-
 .../cassandra/exceptions/CDCWriteException.java |  26 ++
 .../cassandra/exceptions/ExceptionCode.java |  19 +-
 .../transport/messages/ErrorMessage.java|   5 +
 test/conf/cassandra.yaml|   2 +
 test/conf/cdc.yaml  |   3 +
 .../cassandra/db/commitlog/CDCTestReplayer.java |  76 
 .../CommitLogSegmentManagerCDCTest.java | 351 +++
 21 files changed, 617 insertions(+), 161 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9da8572/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8f6fa48..cdcfe25 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -25,6 +25,7 @@
  * Upgrade SLF4J from 1.7.7 to 1.7.25 (CASSANDRA-12996)
  * Default for start_native_transport now true if not set in config 
(CASSANDRA-13656)
  * Don't add localhost to the graph when calculating where to stream from 
(CASSANDRA-13583)
+ * Make CDC availability more deterministic via hard-linking (CASSANDRA-12148)
  * Allow skipping equality-restricted clustering columns in ORDER BY clause 
(CASSANDRA-10271)
  * Use common nowInSec for validation compactions (CASSANDRA-13671)
  * Improve handling of IR prepare failures (CASSANDRA-13672)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9da8572/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 253d773..2038342 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -25,7 +25,11 @@ New features
- Support for arithmetic operations on number has been added. See 
CASSANDRA-11935
- Preview expected streaming required for a repair (nodetool repair 
--preview), and validate the
  consistency of repaired data between nodes (nodetool repair --validate). 
See CASSANDRA-13257
-   - Support for selecting Map values and Set elements has been added for 
SELECT queries. See CASSANDRA-7396 
+   - Support for selecting Map values and Set elements has been added for 
SELECT queries. See CASSANDRA-7396
+   - Change-Data-Capture has been modified to make CommitLogSegments available
+ immediately upon creation via hard-linking the files. This means that 
incomplete
+ segments will be available in cdc_raw rather than fully flushed. See 
documentation
+ and CASSANDRA-12148 for more detail.
 
 Upgrading
 -

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e9da8572/doc/source/operating/cdc.rst
--
diff --git a/doc/source/operating/cdc.rst b/doc/source/operating/cdc.rst
index 192f62a..a7177b5 100644
--- a/doc/source/operating/cdc.rst
+++ b/doc/source/operating/cdc.rst
@@ -23,18 +23,26 @@ Overview
 
 
 Change data capture (CDC) provides a mechanism to flag specific tables for 
archival as well as rejecting writes to those
-tables once a configurable size-on-disk for the combined flushed and unflushed 
CDC-log is reached. An operator can
-enable CDC on a table by setting the table property ``cdc=true`` (either when 
:ref:`creating the table
-` or :ref:`altering it `), 
after which any CommitLogSegments containing
-data for a CDC-enabled table are moved to the directory specified in 
``cassandra.yaml`` on 

[jira] [Updated] (CASSANDRA-12148) Improve determinism of CDC data availability

[Adam Holmberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Holmberg updated CASSANDRA-12148:
--
Labels: client-impacting native_protocol  (was: )

> Improve determinism of CDC data availability
> 
>
> Key: CASSANDRA-12148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12148
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>  Labels: client-impacting, native_protocol
> Fix For: 4.x
>
>
> The latency with which CDC data becomes available has a known limitation due 
> to our reliance on CommitLogSegments being discarded to have the data 
> available in cdc_raw: if a slowly written table co-habitates a 
> CommitLogSegment with CDC data, the CommitLogSegment won't be flushed until 
> we hit either memory pressure on memtables or CommitLog limit pressure. 
> Ultimately, this leaves a non-deterministic element to when data becomes 
> available for CDC consumption unless a consumer parses live CommitLogSegments.
> To work around this limitation and make semi-realtime CDC consumption more 
> friendly to end-users, I propose we extend CDC as follows:
> h6. High level:
> * Consumers parse hard links of active CommitLogSegments in cdc_raw instead 
> of waiting for flush/discard and file move
> * C* stores an offset of the highest seen CDC mutation in a separate idx file 
> per commit log segment in cdc_raw. Clients tail this index file, delta their 
> local last parsed offset on change, and parse the corresponding commit log 
> segment using their last parsed offset as min
> * C* flags that index file with an offset and DONE when the file is flushed 
> so clients know when they can clean up
> h6. Details:
> * On creation of a CommitLogSegment, also hard-link the file in cdc_raw
> * On first write of a CDC-enabled mutation to a segment, we:
> ** Flag it as {{CDCState.CONTAINS}}
> ** Set a long tracking the {{CommitLogPosition}} of the 1st CDC-enabled 
> mutation in the log
> ** Set a long in the CommitLogSegment tracking the offset of the end of the 
> last written CDC mutation in the segment if higher than the previously known 
> highest CDC offset
> * On subsequent writes to the segment, we update the offset of the highest 
> known CDC data
> * On CommitLogSegment fsync, we write a file in cdc_raw as 
> _cdc.idx containing the min offset and end offset fsynced to 
> disk per file
> * On segment discard, if CDCState == {{CDCState.PERMITTED}}, delete both the 
> segment in commitlog and in cdc_raw
> * On segment discard, if CDCState == {{CDCState.CONTAINS}}, delete the 
> segment in commitlog and update the _cdc.idx file w/end offset 
> and a DONE marker
> * On segment replay, store the highest end offset of seen CDC-enabled 
> mutations from a segment and write that to _cdc.idx on 
> completion of segment replay. This should bridge the potential correctness 
> gap of a node writing to a segment and then dying before it can write the 
> _cdc.idx file.
> This should allow clients to skip the beginning of a file to the 1st CDC 
> mutation, track an offset of how far they've parsed, delta against the 
> _cdc.idx file end offset, and use that as a determinant on when to parse new 
> CDC data. Any existing clients written to the initial implementation of CDC 
> need only add the _cdc.idx logic and checking for DONE marker 
> to their code, so the burden on users to update to support this should be 
> quite small for the benefit of having data available as soon as it's fsynced 
> instead of at a non-deterministic time when potentially unrelated tables are 
> flushed.
> Finally, we should look into extending the interface on CommitLogReader to be 
> more friendly for realtime parsing, perhaps supporting taking a 
> CommitLogDescriptor and RandomAccessReader and resuming readSection calls, 
> assuming the reader is at the start of a SyncSegment. Would probably also 
> need to rewind to the start of the segment before returning so subsequent 
> calls would respect this contract. This would skip needing to deserialize the 
> descriptor and all completed SyncSegments to get to the root of the desired 
> segment for parsing.
> One alternative we discussed offline - instead of just storing the highest 
> seen CDC offset, we could instead store an offset per CDC mutation 
> (potentially delta encoded) in the idx file to allow clients to seek and only 
> parse the mutations with CDC enabled. My hunch is that the performance delta 
> from doing so wouldn't justify the complexity given the SyncSegment 
> deserialization and seeking restrictions in the compressed and encrypted 
> cases as mentioned above.
> The only complication I can think of with the above design is uncompressed 
> mmapped CommitLogSegments on 

[jira] [Updated] (CASSANDRA-13789) Reduce memory copies and object creations when acting on ByteBufs

[Norman Maurer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Norman Maurer updated CASSANDRA-13789:
--
Attachment: 0001-CBUtil.sizeOfLongString-encodes-String-to-byte-to-ca.patch

And another one...

> Reduce memory copies and object creations when acting on  ByteBufs
> --
>
> Key: CASSANDRA-13789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13789
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Norman Maurer
>Assignee: Norman Maurer
> Attachments: 
> 0001-CBUtil.sizeOfLongString-encodes-String-to-byte-to-ca.patch, 
> 0001-Reduce-memory-copies-and-object-creations-when-actin.patch
>
>
> There are multiple "low-hanging-fruits" when it comes to reduce memory copies 
> and object allocations when acting on ByteBufs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13776) Adding a field to an UDT can corrupte the tables using it

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13776:
---
   Resolution: Fixed
Fix Version/s: 4.0
   3.11.1
   3.0.15
   Status: Resolved  (was: Ready to Commit)

Committed into 3.0 at cf0b6d107bade419dada49a5da40d2579c80ade8 and merged into 
3.11 and trunk

> Adding a field to an UDT can corrupte the tables using it
> -
>
> Key: CASSANDRA-13776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13776
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Critical
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> Adding a field to an UDT which is used as a {{Set}} element or as a {{Map}} 
> element can corrupt the table.
> The problem can be reproduced using the following test case:
> {code}
> @Test
> public void testReadAfterAlteringUserTypeNestedWithinSet() throws 
> Throwable
> {
> String ut1 = createType("CREATE TYPE %s (a int)");
> String columnType = KEYSPACE + "." + ut1;
> try
> {
> createTable("CREATE TABLE %s (x int PRIMARY KEY, y set columnType + ">>)");
> disableCompaction();
> execute("INSERT INTO %s (x, y) VALUES(1, ?)", set(userType(1), 
> userType(2)));
> assertRows(execute("SELECT * FROM %s"), row(1, set(userType(1), 
> userType(2;
> flush();
> assertRows(execute("SELECT * FROM %s WHERE x = 1"),
>row(1, set(userType(1), userType(2;
> execute("ALTER TYPE " + KEYSPACE + "." + ut1 + " ADD b int");
> execute("UPDATE %s SET y = y + ? WHERE x = 1",
> set(userType(1, 1), userType(1, 2), userType(2, 1)));
> flush();
> assertRows(execute("SELECT * FROM %s WHERE x = 1"),
>row(1, set(userType(1),
>   userType(1, 1),
>   userType(1, 2),
>   userType(2),
>   userType(2, 1;
> compact();
> assertRows(execute("SELECT * FROM %s WHERE x = 1"),
>row(1, set(userType(1),
>   userType(1, 1),
>   userType(1, 2),
>   userType(2),
>   userType(2, 1;
> }
> finally
> {
> enableCompaction();
> }
> }
> {code} 
> There are in fact 2 problems:
> # When the {{sets}} from the 2 versions are merged the {{ColumnDefinition}} 
> being picked up can be the older one. In which case when the tuples are 
> sorted it my lead to an {{IndexOutOfBoundsException}}.
> # During compaction, the old column definition can be the one being kept for 
> the SSTable metadata. If it is the case the SSTable will not be readable any 
> more and will be marked as {{corrupted}}.
> If one of the tables using the type has a Materialized View attached to it, 
> the MV updates can also fail with {{IndexOutOfBoundsException}}.
> This problem can be reproduced using the following test:
> {code}
> @Test
> public void testAlteringUserTypeNestedWithinSetWithView() throws Throwable
> {
> String columnType = typeWithKs(createType("CREATE TYPE %s (a int)"));
> createTable("CREATE TABLE %s (pk int, c int, v int, s set columnType + ">>, PRIMARY KEY (pk, c))");
> execute("CREATE MATERIALIZED VIEW " + keyspace() + ".view1 AS SELECT 
> c, pk, v FROM %s WHERE pk IS NOT NULL AND c IS NOT NULL AND v IS NOT NULL 
> PRIMARY KEY (c, pk)");
> execute("INSERT INTO %s (pk, c, v, s) VALUES(?, ?, ?, ?)", 1, 1, 1, 
> set(userType(1), userType(2)));
> flush();
> execute("ALTER TYPE " + columnType + " ADD b int");
> execute("UPDATE %s SET s = s + ?, v = ? WHERE pk = ? AND c = ?",
> set(userType(1, 1), userType(1, 2), userType(2, 1)), 2, 1, 1);
> assertRows(execute("SELECT * FROM %s WHERE pk = ? AND c = ?", 1, 1),
>row(1, 1, 2, set(userType(1),
> userType(1, 1),
> userType(1, 2),
> userType(2),
> userType(2, 1;
> }
> {code}  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: 

cassandra-builds git commit: Set build retention to 50

[mshuler]

Repository: cassandra-builds
Updated Branches:
  refs/heads/master b0f3195e4 -> 0179814f4


Set build retention to 50


Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/0179814f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/0179814f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/0179814f

Branch: refs/heads/master
Commit: 0179814f4544106ddb5e33966ed3679270b2a044
Parents: b0f3195
Author: Michael Shuler 
Authored: Thu Aug 24 11:32:22 2017 -0500
Committer: Michael Shuler 
Committed: Thu Aug 24 11:32:22 2017 -0500

--
 jenkins-dsl/cassandra_job_dsl_seed.groovy | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/0179814f/jenkins-dsl/cassandra_job_dsl_seed.groovy
--
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy 
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index d251bf4..2daca85 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -69,7 +69,7 @@ job('Cassandra-template-artifacts') {
 jdk(jdkLabel)
 label(slaveLabel)
 logRotator {
-numToKeep(10)
+numToKeep(50)
 artifactNumToKeep(1)
 }
 wrappers {
@@ -113,7 +113,7 @@ job('Cassandra-template-test') {
 jdk(jdkLabel)
 label(slaveLabel)
 logRotator {
-numToKeep(10)
+numToKeep(50)
 }
 wrappers {
 timeout {
@@ -163,7 +163,7 @@ job('Cassandra-template-dtest') {
 jdk(jdkLabel)
 label(slaveLabel)
 logRotator {
-numToKeep(10)
+numToKeep(50)
 }
 wrappers {
 timeout {
@@ -209,7 +209,7 @@ matrixJob('Cassandra-template-cqlsh-tests') {
 disabled(true)
 description(jobDescription)
 logRotator {
-numToKeep(10)
+numToKeep(50)
 }
 wrappers {
 timeout {
@@ -383,7 +383,7 @@ job('Cassandra-devbranch-testall') {
 jdk(jdkLabel)
 label(slaveLabel)
 logRotator {
-numToKeep(10)
+numToKeep(50)
 }
 wrappers {
 timeout {
@@ -435,7 +435,7 @@ job('Cassandra-devbranch-dtest') {
 jdk(jdkLabel)
 label(slaveLabel)
 logRotator {
-numToKeep(10)
+numToKeep(50)
 }
 wrappers {
 timeout {
@@ -486,7 +486,7 @@ matrixJob('Cassandra-devbranch-cqlsh-tests') {
 description(jobDescription)
 concurrentBuild()
 logRotator {
-numToKeep(10)
+numToKeep(50)
 }
 wrappers {
 timeout {


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140263#comment-16140263
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13740:
---

Thanks for the review comments, I will change it and send it updated one soon.

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140263#comment-16140263
 ] 

Jaydeepkumar Chovatia edited comment on CASSANDRA-13740 at 8/24/17 4:29 PM:


Thanks for the review comments, I will change it and send updated one soon.


was (Author: chovatia.jayd...@gmail.com):
Thanks for the review comments, I will change it and send it updated one soon.

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13772) Add a skip read validation flag to cassandra-stress

[Eduard Tudenhoefner (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140255#comment-16140255
 ] 

Eduard Tudenhoefner commented on CASSANDRA-13772:
-

did a quick look over the code and changes lgtm

> Add a skip read validation flag to cassandra-stress
> ---
>
> Key: CASSANDRA-13772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13772
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Noam Hasson
>Assignee: Noam Hasson
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: 13772-trunk.txt
>
>
> When running cassandra-stress with read operations, you must make sure all 
> the data was populated beforehand or else you will get the following errors:
> java.io.IOException: Operation x0 on key(s) [4d31314e32314b395030]: Data 
> returned was not validated
>   at org.apache.cassandra.stress.Operation.error(Operation.java:127)
>   at 
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:91)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:245)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:453)
> java.lang.RuntimeException: Failed to execute warmup
>   at 
> org.apache.cassandra.stress.StressAction.warmup(StressAction.java:117)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:62)
>   at org.apache.cassandra.stress.Stress.run(Stress.java:143)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Even if you use the "-errors ignore" flag you'll get a lot of the following 
> messages which will both slow the stress and prevent it from displaying the 
> metrics:
> Operation x0 on key(s) [4b3539393831374e3431]: Data returned was not validated
> Operation x0 on key(s) [4f4b3936363233375030]: Data returned was not validated
> Operation x0 on key(s) [4d304c4b32384f4b3031]: Data returned was not validated
> What I propose is to add a flag to skip the read validation, such as:
> -errors skip-read-validations
> This way when needed you can run a mixed workload and ignore validation of 
> unpopulated data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[8/8] cassandra git commit: Merge branch cassandra-3.11 into trunk

[blerer]

Merge branch cassandra-3.11 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4838e81a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4838e81a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4838e81a

Branch: refs/heads/trunk
Commit: 4838e81a64eb118d0f86463b61b43c450a199dc8
Parents: 41ef972 8d98a99
Author: Benjamin Lerer 
Authored: Thu Aug 24 18:23:02 2017 +0200
Committer: Benjamin Lerer 
Committed: Thu Aug 24 18:24:02 2017 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/SerializationHeader.java   |  13 +-
 .../db/rows/AbstractTypeVersionComparator.java  | 121 
 src/java/org/apache/cassandra/db/rows/Row.java  |  18 +-
 src/java/org/apache/cassandra/db/rows/Rows.java |  20 +-
 .../io/sstable/format/SSTableReader.java|  20 +-
 .../org/apache/cassandra/cql3/CQLTester.java|  20 +-
 .../cql3/validation/entities/UserTypesTest.java | 151 +++
 .../rows/AbstractTypeVersionComparatorTest.java | 188 +++
 9 files changed, 532 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4838e81a/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4838e81a/src/java/org/apache/cassandra/db/SerializationHeader.java
--
diff --cc src/java/org/apache/cassandra/db/SerializationHeader.java
index 1f937f8,ddf83b7..4266629
--- a/src/java/org/apache/cassandra/db/SerializationHeader.java
+++ b/src/java/org/apache/cassandra/db/SerializationHeader.java
@@@ -84,8 -84,9 +84,9 @@@ public class SerializationHeade
  // our stats merging on the compacted files headers, which as we just 
said can be somewhat inaccurate,
  // but rather on their stats stored in StatsMetadata that are fully 
accurate.
  EncodingStats.Collector stats = new EncodingStats.Collector();
 -PartitionColumns.Builder columns = PartitionColumns.builder();
 -// We need to order the SSTables by descending generation to be sure 
that we use latest column definitions.
 +RegularAndStaticColumns.Builder columns = 
RegularAndStaticColumns.builder();
- for (SSTableReader sstable : sstables)
++// We need to order the SSTables by descending generation to be sure 
that we use latest column metadata.
+ for (SSTableReader sstable : orderByDescendingGeneration(sstables))
  {
  stats.updateTimestamp(sstable.getMinTimestamp());
  stats.updateLocalDeletionTime(sstable.getMinLocalDeletionTime());
@@@ -95,9 -99,19 +96,19 @@@
  return new SerializationHeader(true, metadata, columns.build(), 
stats.get());
  }
  
+ private static Collection 
orderByDescendingGeneration(Collection sstables)
+ {
+ if (sstables.size() < 2)
+ return sstables;
+ 
+ List readers = new ArrayList<>(sstables);
+ readers.sort(SSTableReader.generationReverseComparator);
+ return readers;
+ }
+ 
  public SerializationHeader(boolean isForSSTable,
 -   CFMetaData metadata,
 -   PartitionColumns columns,
 +   TableMetadata metadata,
 +   RegularAndStaticColumns columns,
 EncodingStats stats)
  {
  this(isForSSTable,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4838e81a/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --cc src/java/org/apache/cassandra/db/rows/Row.java
index 7449e51,60ee4d4..5999864
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@@ -721,10 -721,25 +721,25 @@@ public interface Row extends Unfiltered
  
  public void reduce(int idx, ColumnData data)
  {
- column = data.column();
 -if (useColumnDefinition(data.column()))
++if (useColumnMetadata(data.column()))
+ column = data.column();
+ 
  versions.add(data);
  }
  
+ /**
 - * Determines it the {@code ColumnDefinition} is the one that 
should be used.
 - * @param dataColumn the {@code ColumnDefinition} to use.
 - * @return {@code true} if the {@code ColumnDefinition} is the 
one that should be used, {@code false} otherwise.
++ * Determines it the {@code ColumnMetadata} is the one that 
should be used.
++ * @param dataColumn the {@code 

[4/8] cassandra git commit: Fix the merging of cells with different user type versions

[blerer]

Fix the merging of cells with different user type versions

patch by Benjamin Lerer; reviewed by Robert Stupp  for CASSANDRA-13776


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf0b6d10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf0b6d10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf0b6d10

Branch: refs/heads/trunk
Commit: cf0b6d107bade419dada49a5da40d2579c80ade8
Parents: 1a2ad2e
Author: Benjamin Lerer 
Authored: Thu Aug 24 18:19:28 2017 +0200
Committer: Benjamin Lerer 
Committed: Thu Aug 24 18:19:28 2017 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/SerializationHeader.java   |  13 +-
 .../cassandra/db/marshal/AbstractType.java  |  10 +
 .../apache/cassandra/db/marshal/TupleType.java  |   5 +
 .../apache/cassandra/db/marshal/UserType.java   |  10 +
 .../db/rows/AbstractTypeVersionComparator.java  | 121 
 src/java/org/apache/cassandra/db/rows/Row.java  |  18 +-
 src/java/org/apache/cassandra/db/rows/Rows.java |  21 ++-
 .../io/sstable/format/SSTableReader.java|  23 +--
 .../org/apache/cassandra/cql3/CQLTester.java|  20 +-
 .../cql3/validation/entities/UserTypesTest.java | 151 +++
 .../rows/AbstractTypeVersionComparatorTest.java | 184 +++
 12 files changed, 555 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a35cc1b..ba35152 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Fix the merging of cells with different user type versions (CASSANDRA-13776)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/SerializationHeader.java
--
diff --git a/src/java/org/apache/cassandra/db/SerializationHeader.java 
b/src/java/org/apache/cassandra/db/SerializationHeader.java
index 19dad95..494c2a3 100644
--- a/src/java/org/apache/cassandra/db/SerializationHeader.java
+++ b/src/java/org/apache/cassandra/db/SerializationHeader.java
@@ -108,7 +108,8 @@ public class SerializationHeader
 // but rather on their stats stored in StatsMetadata that are fully 
accurate.
 EncodingStats.Collector stats = new EncodingStats.Collector();
 PartitionColumns.Builder columns = PartitionColumns.builder();
-for (SSTableReader sstable : sstables)
+// We need to order the SSTables by descending generation to be sure 
that we use latest column definitions.
+for (SSTableReader sstable : orderByDescendingGeneration(sstables))
 {
 stats.updateTimestamp(sstable.getMinTimestamp());
 stats.updateLocalDeletionTime(sstable.getMinLocalDeletionTime());
@@ -121,6 +122,16 @@ public class SerializationHeader
 return new SerializationHeader(true, metadata, columns.build(), 
stats.get());
 }
 
+private static Collection 
orderByDescendingGeneration(Collection sstables)
+{
+if (sstables.size() < 2)
+return sstables;
+
+List readers = new ArrayList<>(sstables);
+readers.sort(SSTableReader.generationReverseComparator);
+return readers;
+}
+
 public SerializationHeader(boolean isForSSTable,
CFMetaData metadata,
PartitionColumns columns,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/marshal/AbstractType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java 
b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
index 77e0971..20062bd 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
@@ -325,6 +325,16 @@ public abstract class AbstractType implements 
Comparator
 return false;
 }
 
+public boolean isTuple()
+{
+return false;
+}
+
+public boolean isUDT()
+{
+return false;
+}
+
 public AbstractType freeze()
 {
 return this;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/marshal/TupleType.java
--
diff --git 

[1/8] cassandra git commit: Copy session properties on cqlsh.py do_login

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 1a2ad2ee0 -> cf0b6d107
  refs/heads/cassandra-3.11 07f67 -> 8d98a9923
  refs/heads/trunk 41ef972b1 -> 4838e81a6


Copy session properties on cqlsh.py do_login

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13640


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a2ad2ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a2ad2ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a2ad2ee

Branch: refs/heads/cassandra-3.11
Commit: 1a2ad2ee047a9ed2c5112376585323b27ebfb7b0
Parents: 5e57dd1
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:56:12 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:56:12 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a2ad2ee/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2b49bc3..a35cc1b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
  * Gossip thread slows down when using batch commit log (CASSANDRA-12966)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a2ad2ee/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index c132592..8eb42a3 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2206,6 +2206,12 @@ class Shell(cmd.Cmd):
 else:
 session = conn.connect()
 
+# Copy session properties
+session.default_timeout = self.session.default_timeout
+session.row_factory = self.session.row_factory
+session.default_consistency_level = 
self.session.default_consistency_level
+session.max_trace_wait = self.session.max_trace_wait
+
 # Update after we've connected in case we fail to authenticate
 self.conn = conn
 self.auth_provider = auth_provider


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[7/8] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d98a992
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d98a992
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d98a992

Branch: refs/heads/cassandra-3.11
Commit: 8d98a99231f38102d0ef572ffe948d705169aae4
Parents: 07f cf0b6d1
Author: Benjamin Lerer 
Authored: Thu Aug 24 18:21:08 2017 +0200
Committer: Benjamin Lerer 
Committed: Thu Aug 24 18:22:13 2017 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/SerializationHeader.java   |  13 +-
 .../db/rows/AbstractTypeVersionComparator.java  | 121 
 src/java/org/apache/cassandra/db/rows/Row.java  |  18 +-
 src/java/org/apache/cassandra/db/rows/Rows.java |  22 ++-
 .../io/sstable/format/SSTableReader.java|  20 +-
 .../org/apache/cassandra/cql3/CQLTester.java|  20 +-
 .../cql3/validation/entities/UserTypesTest.java | 151 +++
 .../rows/AbstractTypeVersionComparatorTest.java | 188 +++
 9 files changed, 533 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/CHANGES.txt
--
diff --cc CHANGES.txt
index d39f78c,ba35152..8fb8e2f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
 -3.0.15
 +3.11.1
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Fix the merging of cells with different user type versions 
(CASSANDRA-13776)
   * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
   * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
   * Don't let stress write warmup data if n=0 (CASSANDRA-13773)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/src/java/org/apache/cassandra/db/SerializationHeader.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/src/java/org/apache/cassandra/db/rows/Row.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/src/java/org/apache/cassandra/db/rows/Rows.java
--
diff --cc src/java/org/apache/cassandra/db/rows/Rows.java
index a92bdac,09213a4..3331506
--- a/src/java/org/apache/cassandra/db/rows/Rows.java
+++ b/src/java/org/apache/cassandra/db/rows/Rows.java
@@@ -26,9 -25,7 +26,8 @@@ import org.apache.cassandra.config.CFMe
  import org.apache.cassandra.config.ColumnDefinition;
  import org.apache.cassandra.db.*;
  import org.apache.cassandra.db.partitions.PartitionStatisticsCollector;
- import org.apache.cassandra.db.rows.Row.Deletion;
  import org.apache.cassandra.utils.MergeIterator;
 +import org.apache.cassandra.utils.WrappedInt;
  
  /**
   * Static utilities to work on Row objects.
@@@ -330,78 -312,20 +330,96 @@@ public abstract class Row
  }
  
  /**
 + * Returns a row that is obtained from the given existing row by removing 
everything that is shadowed by data in
 + * the update row. In other words, produces the smallest result row such 
that
 + * {@code merge(result, update, nowInSec) == merge(existing, update, 
nowInSec)} after filtering by rangeDeletion.
 + *
 + * @param existing source row
 + * @param update shadowing row
 + * @param rangeDeletion extra {@code DeletionTime} from covering tombstone
 + * @param nowInSec the current time in seconds (which plays a role during 
reconciliation
 + * because deleted cells always have precedence on timestamp equality and 
deciding if a
 + * cell is a live or not depends on the current time due to expiring 
cells).
 + */
 +public static Row removeShadowedCells(Row existing, Row update, 
DeletionTime rangeDeletion, int nowInSec)
 +{
 +Row.Builder builder = BTreeRow.sortedBuilder();
 +Clustering clustering = existing.clustering();
 +builder.newRow(clustering);
 +
 +DeletionTime deletion = update.deletion().time();
 +if (rangeDeletion.supersedes(deletion))
 +deletion = rangeDeletion;
 +
 +LivenessInfo 

[5/8] cassandra git commit: Fix the merging of cells with different user type versions

[blerer]

Fix the merging of cells with different user type versions

patch by Benjamin Lerer; reviewed by Robert Stupp  for CASSANDRA-13776


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf0b6d10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf0b6d10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf0b6d10

Branch: refs/heads/cassandra-3.0
Commit: cf0b6d107bade419dada49a5da40d2579c80ade8
Parents: 1a2ad2e
Author: Benjamin Lerer 
Authored: Thu Aug 24 18:19:28 2017 +0200
Committer: Benjamin Lerer 
Committed: Thu Aug 24 18:19:28 2017 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/SerializationHeader.java   |  13 +-
 .../cassandra/db/marshal/AbstractType.java  |  10 +
 .../apache/cassandra/db/marshal/TupleType.java  |   5 +
 .../apache/cassandra/db/marshal/UserType.java   |  10 +
 .../db/rows/AbstractTypeVersionComparator.java  | 121 
 src/java/org/apache/cassandra/db/rows/Row.java  |  18 +-
 src/java/org/apache/cassandra/db/rows/Rows.java |  21 ++-
 .../io/sstable/format/SSTableReader.java|  23 +--
 .../org/apache/cassandra/cql3/CQLTester.java|  20 +-
 .../cql3/validation/entities/UserTypesTest.java | 151 +++
 .../rows/AbstractTypeVersionComparatorTest.java | 184 +++
 12 files changed, 555 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a35cc1b..ba35152 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Fix the merging of cells with different user type versions (CASSANDRA-13776)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/SerializationHeader.java
--
diff --git a/src/java/org/apache/cassandra/db/SerializationHeader.java 
b/src/java/org/apache/cassandra/db/SerializationHeader.java
index 19dad95..494c2a3 100644
--- a/src/java/org/apache/cassandra/db/SerializationHeader.java
+++ b/src/java/org/apache/cassandra/db/SerializationHeader.java
@@ -108,7 +108,8 @@ public class SerializationHeader
 // but rather on their stats stored in StatsMetadata that are fully 
accurate.
 EncodingStats.Collector stats = new EncodingStats.Collector();
 PartitionColumns.Builder columns = PartitionColumns.builder();
-for (SSTableReader sstable : sstables)
+// We need to order the SSTables by descending generation to be sure 
that we use latest column definitions.
+for (SSTableReader sstable : orderByDescendingGeneration(sstables))
 {
 stats.updateTimestamp(sstable.getMinTimestamp());
 stats.updateLocalDeletionTime(sstable.getMinLocalDeletionTime());
@@ -121,6 +122,16 @@ public class SerializationHeader
 return new SerializationHeader(true, metadata, columns.build(), 
stats.get());
 }
 
+private static Collection 
orderByDescendingGeneration(Collection sstables)
+{
+if (sstables.size() < 2)
+return sstables;
+
+List readers = new ArrayList<>(sstables);
+readers.sort(SSTableReader.generationReverseComparator);
+return readers;
+}
+
 public SerializationHeader(boolean isForSSTable,
CFMetaData metadata,
PartitionColumns columns,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/marshal/AbstractType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java 
b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
index 77e0971..20062bd 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
@@ -325,6 +325,16 @@ public abstract class AbstractType implements 
Comparator
 return false;
 }
 
+public boolean isTuple()
+{
+return false;
+}
+
+public boolean isUDT()
+{
+return false;
+}
+
 public AbstractType freeze()
 {
 return this;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/marshal/TupleType.java
--
diff --git 

[3/8] cassandra git commit: Fix the merging of cells with different user type versions

[blerer]

Fix the merging of cells with different user type versions

patch by Benjamin Lerer; reviewed by Robert Stupp  for CASSANDRA-13776


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf0b6d10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf0b6d10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf0b6d10

Branch: refs/heads/cassandra-3.11
Commit: cf0b6d107bade419dada49a5da40d2579c80ade8
Parents: 1a2ad2e
Author: Benjamin Lerer 
Authored: Thu Aug 24 18:19:28 2017 +0200
Committer: Benjamin Lerer 
Committed: Thu Aug 24 18:19:28 2017 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/SerializationHeader.java   |  13 +-
 .../cassandra/db/marshal/AbstractType.java  |  10 +
 .../apache/cassandra/db/marshal/TupleType.java  |   5 +
 .../apache/cassandra/db/marshal/UserType.java   |  10 +
 .../db/rows/AbstractTypeVersionComparator.java  | 121 
 src/java/org/apache/cassandra/db/rows/Row.java  |  18 +-
 src/java/org/apache/cassandra/db/rows/Rows.java |  21 ++-
 .../io/sstable/format/SSTableReader.java|  23 +--
 .../org/apache/cassandra/cql3/CQLTester.java|  20 +-
 .../cql3/validation/entities/UserTypesTest.java | 151 +++
 .../rows/AbstractTypeVersionComparatorTest.java | 184 +++
 12 files changed, 555 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a35cc1b..ba35152 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Fix the merging of cells with different user type versions (CASSANDRA-13776)
  * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/SerializationHeader.java
--
diff --git a/src/java/org/apache/cassandra/db/SerializationHeader.java 
b/src/java/org/apache/cassandra/db/SerializationHeader.java
index 19dad95..494c2a3 100644
--- a/src/java/org/apache/cassandra/db/SerializationHeader.java
+++ b/src/java/org/apache/cassandra/db/SerializationHeader.java
@@ -108,7 +108,8 @@ public class SerializationHeader
 // but rather on their stats stored in StatsMetadata that are fully 
accurate.
 EncodingStats.Collector stats = new EncodingStats.Collector();
 PartitionColumns.Builder columns = PartitionColumns.builder();
-for (SSTableReader sstable : sstables)
+// We need to order the SSTables by descending generation to be sure 
that we use latest column definitions.
+for (SSTableReader sstable : orderByDescendingGeneration(sstables))
 {
 stats.updateTimestamp(sstable.getMinTimestamp());
 stats.updateLocalDeletionTime(sstable.getMinLocalDeletionTime());
@@ -121,6 +122,16 @@ public class SerializationHeader
 return new SerializationHeader(true, metadata, columns.build(), 
stats.get());
 }
 
+private static Collection 
orderByDescendingGeneration(Collection sstables)
+{
+if (sstables.size() < 2)
+return sstables;
+
+List readers = new ArrayList<>(sstables);
+readers.sort(SSTableReader.generationReverseComparator);
+return readers;
+}
+
 public SerializationHeader(boolean isForSSTable,
CFMetaData metadata,
PartitionColumns columns,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/marshal/AbstractType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java 
b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
index 77e0971..20062bd 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java
@@ -325,6 +325,16 @@ public abstract class AbstractType implements 
Comparator
 return false;
 }
 
+public boolean isTuple()
+{
+return false;
+}
+
+public boolean isUDT()
+{
+return false;
+}
+
 public AbstractType freeze()
 {
 return this;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf0b6d10/src/java/org/apache/cassandra/db/marshal/TupleType.java
--
diff --git 

[6/8] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d98a992
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d98a992
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d98a992

Branch: refs/heads/trunk
Commit: 8d98a99231f38102d0ef572ffe948d705169aae4
Parents: 07f cf0b6d1
Author: Benjamin Lerer 
Authored: Thu Aug 24 18:21:08 2017 +0200
Committer: Benjamin Lerer 
Committed: Thu Aug 24 18:22:13 2017 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/SerializationHeader.java   |  13 +-
 .../db/rows/AbstractTypeVersionComparator.java  | 121 
 src/java/org/apache/cassandra/db/rows/Row.java  |  18 +-
 src/java/org/apache/cassandra/db/rows/Rows.java |  22 ++-
 .../io/sstable/format/SSTableReader.java|  20 +-
 .../org/apache/cassandra/cql3/CQLTester.java|  20 +-
 .../cql3/validation/entities/UserTypesTest.java | 151 +++
 .../rows/AbstractTypeVersionComparatorTest.java | 188 +++
 9 files changed, 533 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/CHANGES.txt
--
diff --cc CHANGES.txt
index d39f78c,ba35152..8fb8e2f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
 -3.0.15
 +3.11.1
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Fix the merging of cells with different user type versions 
(CASSANDRA-13776)
   * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
   * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
   * Don't let stress write warmup data if n=0 (CASSANDRA-13773)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/src/java/org/apache/cassandra/db/SerializationHeader.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/src/java/org/apache/cassandra/db/rows/Row.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d98a992/src/java/org/apache/cassandra/db/rows/Rows.java
--
diff --cc src/java/org/apache/cassandra/db/rows/Rows.java
index a92bdac,09213a4..3331506
--- a/src/java/org/apache/cassandra/db/rows/Rows.java
+++ b/src/java/org/apache/cassandra/db/rows/Rows.java
@@@ -26,9 -25,7 +26,8 @@@ import org.apache.cassandra.config.CFMe
  import org.apache.cassandra.config.ColumnDefinition;
  import org.apache.cassandra.db.*;
  import org.apache.cassandra.db.partitions.PartitionStatisticsCollector;
- import org.apache.cassandra.db.rows.Row.Deletion;
  import org.apache.cassandra.utils.MergeIterator;
 +import org.apache.cassandra.utils.WrappedInt;
  
  /**
   * Static utilities to work on Row objects.
@@@ -330,78 -312,20 +330,96 @@@ public abstract class Row
  }
  
  /**
 + * Returns a row that is obtained from the given existing row by removing 
everything that is shadowed by data in
 + * the update row. In other words, produces the smallest result row such 
that
 + * {@code merge(result, update, nowInSec) == merge(existing, update, 
nowInSec)} after filtering by rangeDeletion.
 + *
 + * @param existing source row
 + * @param update shadowing row
 + * @param rangeDeletion extra {@code DeletionTime} from covering tombstone
 + * @param nowInSec the current time in seconds (which plays a role during 
reconciliation
 + * because deleted cells always have precedence on timestamp equality and 
deciding if a
 + * cell is a live or not depends on the current time due to expiring 
cells).
 + */
 +public static Row removeShadowedCells(Row existing, Row update, 
DeletionTime rangeDeletion, int nowInSec)
 +{
 +Row.Builder builder = BTreeRow.sortedBuilder();
 +Clustering clustering = existing.clustering();
 +builder.newRow(clustering);
 +
 +DeletionTime deletion = update.deletion().time();
 +if (rangeDeletion.supersedes(deletion))
 +deletion = rangeDeletion;
 +
 +LivenessInfo existingInfo = 

[2/8] cassandra git commit: Copy session properties on cqlsh.py do_login

[blerer]

Copy session properties on cqlsh.py do_login

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13640


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a2ad2ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a2ad2ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a2ad2ee

Branch: refs/heads/trunk
Commit: 1a2ad2ee047a9ed2c5112376585323b27ebfb7b0
Parents: 5e57dd1
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:56:12 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:56:12 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a2ad2ee/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2b49bc3..a35cc1b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
  * Gossip thread slows down when using batch commit log (CASSANDRA-12966)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a2ad2ee/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index c132592..8eb42a3 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2206,6 +2206,12 @@ class Shell(cmd.Cmd):
 else:
 session = conn.connect()
 
+# Copy session properties
+session.default_timeout = self.session.default_timeout
+session.row_factory = self.session.row_factory
+session.default_consistency_level = 
self.session.default_consistency_level
+session.max_trace_wait = self.session.max_trace_wait
+
 # Update after we've connected in case we fail to authenticate
 self.conn = conn
 self.auth_provider = auth_provider


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13578) simplify mx4j configuration

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13578:

Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

committesd sha {{41ef972b181e442f41b53f5cce4def27ceac9e49}}

Thanks!

> simplify mx4j configuration
> ---
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13578) simplify mx4j configuration

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13578:

Fix Version/s: (was: 4.x)
   4.0

> simplify mx4j configuration
> ---
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13796) Fix trigger example on 4.0

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13796:

Fix Version/s: 4.0

> Fix trigger example on 4.0
> --
>
> Key: CASSANDRA-13796
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13796
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
> Fix For: 4.0
>
>
> {{CFMetadata.cfName}} was moved to {{name}} not {{table}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13069) Local batchlog for MV may not be correctly written on node movements

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139795#comment-16139795
 ] 

Paulo Motta edited comment on CASSANDRA-13069 at 8/24/17 4:14 PM:
--

Finally getting back to this after a while, sorry for the delay! Going back to 
the question of the last review round:

{quote}
bq. As far as I can tell, the goal of using a local batchlog is to guarantee 
eventual consistency of a the base table and its views. That is, no matter what 
happens for a given update, either both that update and all the related view 
updates get eventually applied, or none of it is. So I don't understand why:
1. we don't include local view mutations in the batchlog in SP.mutateMV.
2. the base table mutation isn't included in said batchlog alongside it's 
related view updates.
{quote}

I ended up writing a [trunk 
patch|https://github.com/pauloricardomg/cassandra/commit/00a0bef75129a396aeee2cedc89d6a6ec66f610e]
 to include both local and base table mutations in the batchlog as suggested, 
but then looking at the original code I figured that whatever failure happens 
during views update (but before the local base or views are persisted) is 
safeguarded by the [base tablecommit log 
write|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/Keyspace.java#L595]
 prior to the view update, so I don't think we actually need to include the 
local mutations in the batchlog.

Given that remote view writes are [fire and 
forget|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L843],
  the most probable cause of failure during local view writing would be a 
crash, and in that case the commit log replay will re-apply the base mutations 
and views. The two downsides I can think of are:
a) We cannot ensure the commit log is actually persisted before the crash, 
unless batch commit log is used.
b) The user may have durable_writes=false
c) I'm not sure if many other non-crash failures are possible in this case 
(fail local base and view mutations), besides full/corrupted disk, in which 
case you are screwed anyway, but if they happen you'll need to wait until the 
next restart/commitlog replay to have your base-views consistent.

There's not much we can do about a), so it seems we'll just need to live with 
this and rely on repair to fix potential inconsistencies? b) is actually a 
problem even with including local mutations in the batchlog write, unless we 
force batchlog writes to be durable. c) is not a big deal unless there are 
other legitimate non-crash scenarios which I'm not aware of.

Adding the local base mutation to the view batchlog requires the following 
changes:
a) Special case the batchlog write-path to [skip writing the base 
mutation|https://github.com/pauloricardomg/cassandra/commit/00a0bef75129a396aeee2cedc89d6a6ec66f610e#diff-b8ec5deb7141558cf8f868460077be31R94],
 since that will be written by the calling thread after the view updates, and 
[ack 
it|https://github.com/pauloricardomg/cassandra/commit/00a0bef75129a396aeee2cedc89d6a6ec66f610e#diff-9ca8d303d375e01a14d42154eaf46e37R556]
 after the base table write is done so the batchlog can be clean.
b) Special case the batchlog [replay 
path|https://github.com/pauloricardomg/cassandra/commit/00a0bef75129a396aeee2cedc89d6a6ec66f610e#diff-c77d5f1027ee5e8a49e106903a4ca937R458]
 to avoid generating views when replaying base table mutations in the case of 
view batchlogs.

So, unless there's a good reason not mentioned above to include the local 
mutations on the view batchlog, I'd prefer to keep the current approach of 
writing only remote view mutations in the view batchlog. If you agree with 
that, I [added a 
comment|https://github.com/pauloricardomg/cassandra/commit/8314ced9f1077224008e79e4c23d00d3277073d5#diff-71f06c193f5b5e270cf8ac695164f43aR732]
 in the original patch explaining why the local mutations are not included in 
the local batchlog to avoid confusion in the future. Please find the updated 
patch below:

||3.0||trunk||dtest||
|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-13069]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-13069]|[branch|https://github.com/riptano/cassandra-dtest/compare/master...pauloricardomg:13069]|
|[testall|http://jenkins-cassandra.datastax.lan/view/Dev/view/paulomotta/job/pauloricardomg-3.0-13069-testall/lastCompletedBuild/testReport/]|[testall|http://jenkins-cassandra.datastax.lan/view/Dev/view/paulomotta/job/pauloricardomg-trunk-13069-testall/lastCompletedBuild/testReport/]|

cassandra git commit: simplify mx4j configuration

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4b777d627 -> 41ef972b1


simplify mx4j configuration

patch by Jay Zhuang; reviewed by jasobrown for CASSANDRA-13578


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41ef972b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41ef972b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41ef972b

Branch: refs/heads/trunk
Commit: 41ef972b181e442f41b53f5cce4def27ceac9e49
Parents: 4b777d6
Author: Jay Zhuang 
Authored: Tue Jun 6 17:41:56 2017 -0700
Committer: Jason Brown 
Committed: Thu Aug 24 09:13:56 2017 -0700

--
 CHANGES.txt |  1 +
 conf/cassandra-env.sh   | 23 
 .../org/apache/cassandra/utils/Mx4jTool.java| 10 +
 3 files changed, 26 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41ef972b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0a4726b..858312e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * simplify mx4j configuration (Cassandra-13578)
  * Fix trigger example on 4.0 (CASSANDRA-13796)
  * force minumum timeout value (CASSANDRA-9375)
  * use netty for streaming (CASSANDRA-12229)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41ef972b/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 347fbf3..c80e05d 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -282,15 +282,30 @@ JVM_OPTS="$JVM_OPTS 
-Dcom.sun.management.jmxremote.password.file=/etc/cassandra/
 # See http://wiki.apache.org/cassandra/Operations#Monitoring_with_MX4J
 # By default mx4j listens on 0.0.0.0:8081. Uncomment the following lines
 # to control its listen address and port.
-#MX4J_ADDRESS="-Dmx4jaddress=127.0.0.1"
-#MX4J_PORT="-Dmx4jport=8081"
+#MX4J_ADDRESS="127.0.0.1"
+#MX4J_PORT="8081"
 
 # Cassandra uses SIGAR to capture OS metrics CASSANDRA-7838
 # for SIGAR we have to set the java.library.path
 # to the location of the native libraries.
 JVM_OPTS="$JVM_OPTS -Djava.library.path=$CASSANDRA_HOME/lib/sigar-bin"
 
-JVM_OPTS="$JVM_OPTS $MX4J_ADDRESS"
-JVM_OPTS="$JVM_OPTS $MX4J_PORT"
+if [ "x$MX4J_ADDRESS" != "x" ]; then
+if [[ "$MX4J_ADDRESS" == \-Dmx4jaddress* ]]; then
+# Backward compatible with the older style #13578
+JVM_OPTS="$JVM_OPTS $MX4J_ADDRESS"
+else
+JVM_OPTS="$JVM_OPTS -Dmx4jaddress=$MX4J_ADDRESS"
+fi
+fi
+if [ "x$MX4J_PORT" != "x" ]; then
+if [[ "$MX4J_PORT" == \-Dmx4jport* ]]; then
+# Backward compatible with the older style #13578
+JVM_OPTS="$JVM_OPTS $MX4J_PORT"
+else
+JVM_OPTS="$JVM_OPTS -Dmx4jport=$MX4J_PORT"
+fi
+fi
+
 JVM_OPTS="$JVM_OPTS $JVM_EXTRA_OPTS"
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/41ef972b/src/java/org/apache/cassandra/utils/Mx4jTool.java
--
diff --git a/src/java/org/apache/cassandra/utils/Mx4jTool.java 
b/src/java/org/apache/cassandra/utils/Mx4jTool.java
index 41a5b33..cd42aca 100644
--- a/src/java/org/apache/cassandra/utils/Mx4jTool.java
+++ b/src/java/org/apache/cassandra/utils/Mx4jTool.java
@@ -21,6 +21,7 @@ import java.lang.management.ManagementFactory;
 import javax.management.MBeanServer;
 import javax.management.ObjectName;
 
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -76,17 +77,18 @@ public class Mx4jTool
 
 private static String getAddress()
 {
-return System.getProperty("mx4jaddress", 
FBUtilities.getBroadcastAddress().getHostAddress());
+String sAddress = System.getProperty("mx4jaddress");
+if (StringUtils.isEmpty(sAddress))
+sAddress = FBUtilities.getBroadcastAddress().getHostAddress();
+return sAddress;
 }
 
 private static int getPort()
 {
 int port = 8081;
 String sPort = System.getProperty("mx4jport");
-if (sPort != null && !sPort.equals(""))
-{
+if (StringUtils.isNotEmpty(sPort))
 port = Integer.parseInt(sPort);
-}
 return port;
 }
 }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13578) simplify mx4j configuration

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140232#comment-16140232
 ] 

Jason Brown edited comment on CASSANDRA-13578 at 8/24/17 4:13 PM:
--

I'm +1, as well. Will commit today.

Sorry, I also renamed the jira to make it easier to understand


was (Author: jasobrown):
I'm +1, as well. Will commit today.

> simplify mx4j configuration
> ---
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13578) simplify mx4j configuration

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13578:

Summary: simplify mx4j configuration  (was: mx4j configuration minor 
improvement)

> simplify mx4j configuration
> ---
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13578) mx4j configuration minor improvement

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140232#comment-16140232
 ] 

Jason Brown commented on CASSANDRA-13578:
-

I'm +1, as well. Will commit today.

> mx4j configuration minor improvement
> 
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13640) CQLSH error when using 'login' to switch users

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña updated CASSANDRA-13640:
--
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

> CQLSH error when using 'login' to switch users
> --
>
> Key: CASSANDRA-13640
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13640
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Using {{PasswordAuthenticator}} and {{CassandraAuthorizer}}:
> {code}
> bin/cqlsh -u cassandra -p cassandra
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.14-SNAPSHOT | CQL spec 3.4.0 | Native protocol 
> v4]
> Use HELP for help.
> cassandra@cqlsh> create role super with superuser = true and password = 'p' 
> and login = true;
> cassandra@cqlsh> login super;
> Password:
> super@cqlsh> list roles;
> 'Row' object has no attribute 'values'
> {code}
> When we initialize the Shell, we configure certain settings on the session 
> object such as
> {code}
> self.session.default_timeout = request_timeout
> self.session.row_factory = ordered_dict_factory
> self.session.default_consistency_level = cassandra.ConsistencyLevel.ONE
> {code}
> However, once we perform a LOGIN cmd, which calls do_login(..), we create a 
> new cluster/session object but actually never set those settings on the new 
> session.
> It isn't failing on 3.x. 
> As a workaround, it is possible to logout and log back in and things work 
> correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13640) CQLSH error when using 'login' to switch users

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140231#comment-16140231
 ] 

Andrés de la Peña commented on CASSANDRA-13640:
---

Committed as 
[9497191f5bab126c4d83ccbe023554fd6ea95257|https://github.com/apache/cassandra/commit/9497191f5bab126c4d83ccbe023554fd6ea95257].

Dtest committed as 
[da6ad8317e18ebaa5e8b428df79d1da086a19dd9|https://github.com/apache/cassandra-dtest/commit/da6ad8317e18ebaa5e8b428df79d1da086a19dd9].

> CQLSH error when using 'login' to switch users
> --
>
> Key: CASSANDRA-13640
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13640
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Using {{PasswordAuthenticator}} and {{CassandraAuthorizer}}:
> {code}
> bin/cqlsh -u cassandra -p cassandra
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.14-SNAPSHOT | CQL spec 3.4.0 | Native protocol 
> v4]
> Use HELP for help.
> cassandra@cqlsh> create role super with superuser = true and password = 'p' 
> and login = true;
> cassandra@cqlsh> login super;
> Password:
> super@cqlsh> list roles;
> 'Row' object has no attribute 'values'
> {code}
> When we initialize the Shell, we configure certain settings on the session 
> object such as
> {code}
> self.session.default_timeout = request_timeout
> self.session.row_factory = ordered_dict_factory
> self.session.default_consistency_level = cassandra.ConsistencyLevel.ONE
> {code}
> However, once we perform a LOGIN cmd, which calls do_login(..), we create a 
> new cluster/session object but actually never set those settings on the new 
> session.
> It isn't failing on 3.x. 
> As a workaround, it is possible to logout and log back in and things work 
> correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13640) CQLSH error when using 'login' to switch users

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña updated CASSANDRA-13640:
--
Fix Version/s: 4.x
   3.11.x

> CQLSH error when using 'login' to switch users
> --
>
> Key: CASSANDRA-13640
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13640
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Using {{PasswordAuthenticator}} and {{CassandraAuthorizer}}:
> {code}
> bin/cqlsh -u cassandra -p cassandra
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.14-SNAPSHOT | CQL spec 3.4.0 | Native protocol 
> v4]
> Use HELP for help.
> cassandra@cqlsh> create role super with superuser = true and password = 'p' 
> and login = true;
> cassandra@cqlsh> login super;
> Password:
> super@cqlsh> list roles;
> 'Row' object has no attribute 'values'
> {code}
> When we initialize the Shell, we configure certain settings on the session 
> object such as
> {code}
> self.session.default_timeout = request_timeout
> self.session.row_factory = ordered_dict_factory
> self.session.default_consistency_level = cassandra.ConsistencyLevel.ONE
> {code}
> However, once we perform a LOGIN cmd, which calls do_login(..), we create a 
> new cluster/session object but actually never set those settings on the new 
> session.
> It isn't failing on 3.x. 
> As a workaround, it is possible to logout and log back in and things work 
> correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/4] cassandra git commit: Copy session properties on cqlsh.py do_login

[adelapena]

Copy session properties on cqlsh.py do_login

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13640


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a2ad2ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a2ad2ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a2ad2ee

Branch: refs/heads/cassandra-3.0
Commit: 1a2ad2ee047a9ed2c5112376585323b27ebfb7b0
Parents: 5e57dd1
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:56:12 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:56:12 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a2ad2ee/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2b49bc3..a35cc1b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
  * Gossip thread slows down when using batch commit log (CASSANDRA-12966)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a2ad2ee/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index c132592..8eb42a3 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2206,6 +2206,12 @@ class Shell(cmd.Cmd):
 else:
 session = conn.connect()
 
+# Copy session properties
+session.default_timeout = self.session.default_timeout
+session.row_factory = self.session.row_factory
+session.default_consistency_level = 
self.session.default_consistency_level
+session.max_trace_wait = self.session.max_trace_wait
+
 # Update after we've connected in case we fail to authenticate
 self.conn = conn
 self.auth_provider = auth_provider


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/4] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[adelapena]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/07f6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/07f6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/07f6

Branch: refs/heads/trunk
Commit: 07f67cdecac0187a336ce654a14c6f56fc01
Parents: ed9b04d 9497191
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:34:03 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:34:03 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/07f6/CHANGES.txt
--
diff --cc CHANGES.txt
index b22cb5e,936b72c..d39f78c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
 -3.0.15
 +3.11.1
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
 + * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
 + * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
 + * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
 + * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)
 + * Fix digest calculation for counter cells (CASSANDRA-13750)
   * Fix ColumnDefinition.cellValueType() for non-frozen collection and change 
SSTabledump to use type.toJSONString() (CASSANDRA-13573)
   * Skip materialized view addition if the base table doesn't exist 
(CASSANDRA-13737)
   * Drop table should remove corresponding entries in dropped_columns table 
(CASSANDRA-13730)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/07f6/bin/cqlsh.py
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/4] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[adelapena]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b777d62
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b777d62
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b777d62

Branch: refs/heads/trunk
Commit: 4b777d6271b8900f00a23e51aa3d8e42b7a0f34b
Parents: 2e5847d 07f
Author: Andrés de la Peña 
Authored: Thu Aug 24 17:03:41 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 17:03:41 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b777d62/CHANGES.txt
--
diff --cc CHANGES.txt
index 92ff455,d39f78c..0a4726b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -130,8 -6,10 +130,9 @@@
   * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
   * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
  Merged from 3.0:
+  * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
   * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
   * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
 - * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
   * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)
   * Fix digest calculation for counter cells (CASSANDRA-13750)
   * Fix ColumnDefinition.cellValueType() for non-frozen collection and change 
SSTabledump to use type.toJSONString() (CASSANDRA-13573)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b777d62/bin/cqlsh.py
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/4] cassandra git commit: Copy session properties on cqlsh.py do_login

[adelapena]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 5e57dd14e -> 1a2ad2ee0
  refs/heads/trunk 2e5847d29 -> 4b777d627


Copy session properties on cqlsh.py do_login

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13640


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9497191f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9497191f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9497191f

Branch: refs/heads/trunk
Commit: 9497191f5bab126c4d83ccbe023554fd6ea95257
Parents: 3960260
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:29:11 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:29:11 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9497191f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1525289..936b72c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Fix ColumnDefinition.cellValueType() for non-frozen collection and change 
SSTabledump to use type.toJSONString() (CASSANDRA-13573)
  * Skip materialized view addition if the base table doesn't exist 
(CASSANDRA-13737)
  * Drop table should remove corresponding entries in dropped_columns table 
(CASSANDRA-13730)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9497191f/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index c132592..8eb42a3 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2206,6 +2206,12 @@ class Shell(cmd.Cmd):
 else:
 session = conn.connect()
 
+# Copy session properties
+session.default_timeout = self.session.default_timeout
+session.row_factory = self.session.row_factory
+session.default_consistency_level = 
self.session.default_consistency_level
+session.max_trace_wait = self.session.max_trace_wait
+
 # Update after we've connected in case we fail to authenticate
 self.conn = conn
 self.auth_provider = auth_provider


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13772) Add a skip read validation flag to cassandra-stress

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown reassigned CASSANDRA-13772:
---

Assignee: Noam Hasson
Reviewer: Jason Brown

> Add a skip read validation flag to cassandra-stress
> ---
>
> Key: CASSANDRA-13772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13772
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Noam Hasson
>Assignee: Noam Hasson
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: 13772-trunk.txt
>
>
> When running cassandra-stress with read operations, you must make sure all 
> the data was populated beforehand or else you will get the following errors:
> java.io.IOException: Operation x0 on key(s) [4d31314e32314b395030]: Data 
> returned was not validated
>   at org.apache.cassandra.stress.Operation.error(Operation.java:127)
>   at 
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:91)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:99)
>   at 
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:245)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:453)
> java.lang.RuntimeException: Failed to execute warmup
>   at 
> org.apache.cassandra.stress.StressAction.warmup(StressAction.java:117)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:62)
>   at org.apache.cassandra.stress.Stress.run(Stress.java:143)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Even if you use the "-errors ignore" flag you'll get a lot of the following 
> messages which will both slow the stress and prevent it from displaying the 
> metrics:
> Operation x0 on key(s) [4b3539393831374e3431]: Data returned was not validated
> Operation x0 on key(s) [4f4b3936363233375030]: Data returned was not validated
> Operation x0 on key(s) [4d304c4b32384f4b3031]: Data returned was not validated
> What I propose is to add a flag to skip the read validation, such as:
> -errors skip-read-validations
> This way when needed you can run a mixed workload and ignore validation of 
> unpopulated data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra-dtest git commit: Add test to verify that it is possible to list roles after a successful login (CASSANDRA-13640)

[adelapena]

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master 1a0e26603 -> da6ad8317


Add test to verify that it is possible to list roles after a successful login 
(CASSANDRA-13640)


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/da6ad831
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/da6ad831
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/da6ad831

Branch: refs/heads/master
Commit: da6ad8317e18ebaa5e8b428df79d1da086a19dd9
Parents: 1a0e266
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:47:58 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:47:58 2017 +0100

--
 cqlsh_tests/cqlsh_tests.py | 16 
 1 file changed, 16 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/da6ad831/cqlsh_tests/cqlsh_tests.py
--
diff --git a/cqlsh_tests/cqlsh_tests.py b/cqlsh_tests/cqlsh_tests.py
index 418b9f7..8b66a53 100644
--- a/cqlsh_tests/cqlsh_tests.py
+++ b/cqlsh_tests/cqlsh_tests.py
@@ -2032,3 +2032,19 @@ class CqlLoginTest(Tester):
 cqlsh_options=['-u', 'cassandra', '-p', 'cassandra'])
 self.assertEqual([x for x in cqlsh_stdout.split() if x], ['ks1table'])
 self.assert_login_not_allowed('user1', cqlsh_stderr)
+
+def test_list_roles_after_login(self):
+"""
+@jira_ticket CASSANDRA-13640
+
+Verifies that it is possible to list roles after a successful login.
+"""
+out, err, _ = self.node1.run_cqlsh(
+'''
+CREATE ROLE super WITH superuser = true AND password = 'p' AND 
login = true;
+LOGIN super 'p';
+LIST ROLES;
+''',
+cqlsh_options=['-u', 'cassandra', '-p', 'cassandra'])
+self.assertTrue('super' in out)
+self.assertEqual('', err)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[adelapena]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/07f6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/07f6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/07f6

Branch: refs/heads/cassandra-3.11
Commit: 07f67cdecac0187a336ce654a14c6f56fc01
Parents: ed9b04d 9497191
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:34:03 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:34:03 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/07f6/CHANGES.txt
--
diff --cc CHANGES.txt
index b22cb5e,936b72c..d39f78c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
 -3.0.15
 +3.11.1
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
 + * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
 + * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
 + * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
 + * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)
 + * Fix digest calculation for counter cells (CASSANDRA-13750)
   * Fix ColumnDefinition.cellValueType() for non-frozen collection and change 
SSTabledump to use type.toJSONString() (CASSANDRA-13573)
   * Skip materialized view addition if the base table doesn't exist 
(CASSANDRA-13737)
   * Drop table should remove corresponding entries in dropped_columns table 
(CASSANDRA-13730)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/07f6/bin/cqlsh.py
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/2] cassandra git commit: Copy session properties on cqlsh.py do_login

[adelapena]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 ed9b04d6a -> 07f67


Copy session properties on cqlsh.py do_login

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13640


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9497191f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9497191f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9497191f

Branch: refs/heads/cassandra-3.11
Commit: 9497191f5bab126c4d83ccbe023554fd6ea95257
Parents: 3960260
Author: Andrés de la Peña 
Authored: Thu Aug 24 16:29:11 2017 +0100
Committer: Andrés de la Peña 
Committed: Thu Aug 24 16:29:11 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9497191f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1525289..936b72c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
  * Fix ColumnDefinition.cellValueType() for non-frozen collection and change 
SSTabledump to use type.toJSONString() (CASSANDRA-13573)
  * Skip materialized view addition if the base table doesn't exist 
(CASSANDRA-13737)
  * Drop table should remove corresponding entries in dropped_columns table 
(CASSANDRA-13730)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9497191f/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index c132592..8eb42a3 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2206,6 +2206,12 @@ class Shell(cmd.Cmd):
 else:
 session = conn.connect()
 
+# Copy session properties
+session.default_timeout = self.session.default_timeout
+session.row_factory = self.session.row_factory
+session.default_consistency_level = 
self.session.default_consistency_level
+session.max_trace_wait = self.session.max_trace_wait
+
 # Update after we've connected in case we fail to authenticate
 self.conn = conn
 self.auth_provider = auth_provider


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13796) Fix trigger example on 4.0

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown resolved CASSANDRA-13796.
-
Resolution: Fixed
  Reviewer: Jason Brown

Committed as sha {{2e5847d29bbdd45fd4fc73f071779d91326ceeba}}. Thanks!

This closes #140

> Fix trigger example on 4.0
> --
>
> Key: CASSANDRA-13796
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13796
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>
> {{CFMetadata.cfName}} was moved to {{name}} not {{table}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Fix trigger example on 4.0

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 652d9f64f -> 2e5847d29


Fix trigger example on 4.0

patch by Chris Lohfink; reviewed by jasobrown for CASSANDRA-13796


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e5847d2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e5847d2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e5847d2

Branch: refs/heads/trunk
Commit: 2e5847d29bbdd45fd4fc73f071779d91326ceeba
Parents: 652d9f6
Author: Chris Lohfink 
Authored: Thu Aug 24 09:55:55 2017 -0500
Committer: Jason Brown 
Committed: Thu Aug 24 08:34:34 2017 -0700

--
 CHANGES.txt| 1 +
 .../triggers/src/org/apache/cassandra/triggers/AuditTrigger.java   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e5847d2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e19e9dd..92ff455 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Fix trigger example on 4.0 (CASSANDRA-13796)
  * force minumum timeout value (CASSANDRA-9375)
  * use netty for streaming (CASSANDRA-12229)
  * Use netty for internode messaging (CASSANDRA-8457)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e5847d2/examples/triggers/src/org/apache/cassandra/triggers/AuditTrigger.java
--
diff --git 
a/examples/triggers/src/org/apache/cassandra/triggers/AuditTrigger.java 
b/examples/triggers/src/org/apache/cassandra/triggers/AuditTrigger.java
index 2528ec7..b0172b0 100644
--- a/examples/triggers/src/org/apache/cassandra/triggers/AuditTrigger.java
+++ b/examples/triggers/src/org/apache/cassandra/triggers/AuditTrigger.java
@@ -45,7 +45,7 @@ public class AuditTrigger implements ITrigger
 
 audit.row()
  .add("keyspace_name", update.metadata().keyspace)
- .add("table_name", update.metadata().table)
+ .add("table_name", update.metadata().name)
  .add("primary_key", 
update.metadata().partitionKeyType.getString(update.partitionKey().getKey()));
 
 return Collections.singletonList(audit.buildAsMutation());


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13796) Fix trigger example on 4.0

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13796:

Summary: Fix trigger example on 4.0  (was: Trigger example wont compile)

> Fix trigger example on 4.0
> --
>
> Key: CASSANDRA-13796
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13796
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>
> {{CFMetadata.cfName}} was moved to {{name}} not {{table}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13796) Trigger example wont compile

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140125#comment-16140125
 ] 

ASF GitHub Bot commented on CASSANDRA-13796:


GitHub user clohfink opened a pull request:

https://github.com/apache/cassandra/pull/140

Fix trigger example for CASSANDRA-13796



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/clohfink/cassandra 13796

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cassandra/pull/140.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #140






> Trigger example wont compile
> 
>
> Key: CASSANDRA-13796
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13796
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>
> {{CFMetadata.cfName}} was moved to {{name}} not {{table}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13796) Trigger example wont compile

[Chris Lohfink (JIRA)]

Chris Lohfink created CASSANDRA-13796:
-

 Summary: Trigger example wont compile
 Key: CASSANDRA-13796
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13796
 Project: Cassandra
  Issue Type: Bug
Reporter: Chris Lohfink
Assignee: Chris Lohfink


{{CFMetadata.cfName}} was moved to {{name}} not {{table}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140112#comment-16140112
 ] 

Paulo Motta commented on CASSANDRA-11713:
-

Hmm the context I proposed this was during investigation of CASSANDRA-11363, 
which was solved, so I think it's fine to close this for now and reopen in case 
another need arises to investigate hairy conditions on the NTR pool.

> Add ability to log thread dump when NTR pool is blocked
> ---
>
> Key: CASSANDRA-11713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11713
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Minor
> Attachments: ThreadDumper.png
>
>
> Thread dumps are very useful for troubleshooting Native-Transport-Requests 
> contention issues like CASSANDRA-11363 and CASSANDRA-11529.
> While they could be generated externally with {{jstack}}, sometimes the 
> conditions are transient and it's hard to catch the exact moment when they 
> happen, so it could be useful to generate and log them upon user request when 
> certain internal condition happens.
> I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} 
> that when enabled via JMX generates and logs a single thread dump on the 
> system log when the thread pool queue is full.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11713:

Resolution: Later
Status: Resolved  (was: Patch Available)

> Add ability to log thread dump when NTR pool is blocked
> ---
>
> Key: CASSANDRA-11713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11713
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Minor
> Attachments: ThreadDumper.png
>
>
> Thread dumps are very useful for troubleshooting Native-Transport-Requests 
> contention issues like CASSANDRA-11363 and CASSANDRA-11529.
> While they could be generated externally with {{jstack}}, sometimes the 
> conditions are transient and it's hard to catch the exact moment when they 
> happen, so it could be useful to generate and log them upon user request when 
> certain internal condition happens.
> I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} 
> that when enabled via JMX generates and logs a single thread dump on the 
> system log when the thread pool queue is full.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13578) mx4j configuration minor improvement

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-13578:

Status: Ready to Commit  (was: Patch Available)

> mx4j configuration minor improvement
> 
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13578) mx4j configuration minor improvement

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140082#comment-16140082
 ] 

Jeremiah Jordan commented on CASSANDRA-13578:
-

+1

> mx4j configuration minor improvement
> 
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13578) mx4j configuration minor improvement

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-13578:

Reviewer: Jeremiah Jordan

> mx4j configuration minor improvement
> 
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13794) Fix short read protection logic for querying more rows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-13794:
--
Reviewer: Benedict

> Fix short read protection logic for querying more rows
> --
>
> Key: CASSANDRA-13794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13794
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Benedict
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x
>
>
> Discovered by [~benedict] while reviewing CASSANDRA-13747:
> {quote}
> While reviewing I got a little suspicious of the modified line 
> {{DataResolver}} :479, as it seemed that n and x were the wrong way around... 
> and, reading the comment of intent directly above, and reproducing the 
> calculation, they are indeed.
> This is probably a significant enough bug that it warrants its own ticket for 
> record keeping, though I'm fairly agnostic on that decision.
> I'm a little concerned about our current short read behaviour, as right now 
> it seems we should be requesting exactly one row, for any size of under-read, 
> which could mean extremely poor performance in case of large under-reads.
> I would suggest that the outer unconditional {{Math.max}} is a bad idea, has 
> been (poorly) insulating us from this error, and that we should first be 
> asserting that the calculation yields a value >= 0 before setting to 1.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11748) Schema version mismatch may leads to Casandra OOM at bootstrap during a rolling upgrade process

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140006#comment-16140006
 ] 

Aleksey Yeschenko commented on CASSANDRA-11748:
---

[~mbyrd] Feel free to drop it on my plate - I have a pretty good picture of 
what I'd like to see here, which should be pretty simple. If not, I'll happily 
review your work instead. For now, cancelling this patch, though. Should we do 
the same for CASSANDRA-13569, given that the fix for both is essentially same?

> Schema version mismatch may leads to Casandra OOM at bootstrap during a 
> rolling upgrade process
> ---
>
> Key: CASSANDRA-11748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11748
> Project: Cassandra
>  Issue Type: Bug
> Environment: Rolling upgrade process from 1.2.19 to 2.0.17. 
> CentOS 6.6
> Occurred in different C* node of different scale of deployment (2G ~ 5G)
>Reporter: Michael Fong
>Assignee: Matt Byrd
>Priority: Critical
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> We have observed multiple times when a multi-node C* (v2.0.17) cluster ran 
> into OOM in bootstrap during a rolling upgrade process from 1.2.19 to 2.0.17. 
> Here is the simple guideline of our rolling upgrade process
> 1. Update schema on a node, and wait until all nodes to be in schema version 
> agreemnt - via nodetool describeclulster
> 2. Restart a Cassandra node
> 3. After restart, there is a chance that the the restarted node has different 
> schema version.
> 4. All nodes in cluster start to rapidly exchange schema information, and any 
> of node could run into OOM. 
> The following is the system.log that occur in one of our 2-node cluster test 
> bed
> --
> Before rebooting node 2:
> Node 1: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,326 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> Node 2: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,122 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> After rebooting node 2, 
> Node 2: DEBUG [main] 2016-04-19 11:18:18,016 MigrationManager.java (line 328) 
> Gossiping my schema version f5270873-ba1f-39c7-ab2e-a86db868b09b
> The node2  keeps submitting the migration task over 100+ times to the other 
> node.
> INFO [GossipStage:1] 2016-04-19 11:18:18,261 Gossiper.java (line 1011) Node 
> /192.168.88.33 has restarted, now UP
> INFO [GossipStage:1] 2016-04-19 11:18:18,262 TokenMetadata.java (line 414) 
> Updating topology for /192.168.88.33
> ...
> DEBUG [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 
> 102) Submitting migration task for /192.168.88.33
> ... ( over 100+ times)
> --
> On the otherhand, Node 1 keeps updating its gossip information, followed by 
> receiving and submitting migrationTask afterwards: 
> INFO [RequestResponseStage:3] 2016-04-19 11:18:18,333 Gossiper.java (line 
> 978) InetAddress /192.168.88.34 is now UP
> ...
> DEBUG [MigrationStage:1] 2016-04-19 11:18:18,496 
> MigrationRequestVerbHandler.java (line 41) Received migration request from 
> /192.168.88.34.
> …… ( over 100+ times)
> DEBUG [OptionalTasks:1] 2016-04-19 11:19:18,337 MigrationManager.java (line 
> 127) submitting migration task for /192.168.88.34
> .  (over 50+ times)
> On the side note, we have over 200+ column families defined in Cassandra 
> database, which may related to this amount of rpc traffic.
> P.S.2 The over requested schema migration task will eventually have 
> InternalResponseStage performing schema merge operation. Since this operation 
> requires a compaction for each merge and is much slower to consume. Thus, the 
> back-pressure of incoming schema migration content objects consumes all of 
> the heap space and ultimately ends up OOM!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-11748) Schema version mismatch may leads to Casandra OOM at bootstrap during a rolling upgrade process

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11748:
--
Status: Open  (was: Patch Available)

> Schema version mismatch may leads to Casandra OOM at bootstrap during a 
> rolling upgrade process
> ---
>
> Key: CASSANDRA-11748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11748
> Project: Cassandra
>  Issue Type: Bug
> Environment: Rolling upgrade process from 1.2.19 to 2.0.17. 
> CentOS 6.6
> Occurred in different C* node of different scale of deployment (2G ~ 5G)
>Reporter: Michael Fong
>Assignee: Matt Byrd
>Priority: Critical
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> We have observed multiple times when a multi-node C* (v2.0.17) cluster ran 
> into OOM in bootstrap during a rolling upgrade process from 1.2.19 to 2.0.17. 
> Here is the simple guideline of our rolling upgrade process
> 1. Update schema on a node, and wait until all nodes to be in schema version 
> agreemnt - via nodetool describeclulster
> 2. Restart a Cassandra node
> 3. After restart, there is a chance that the the restarted node has different 
> schema version.
> 4. All nodes in cluster start to rapidly exchange schema information, and any 
> of node could run into OOM. 
> The following is the system.log that occur in one of our 2-node cluster test 
> bed
> --
> Before rebooting node 2:
> Node 1: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,326 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> Node 2: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,122 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> After rebooting node 2, 
> Node 2: DEBUG [main] 2016-04-19 11:18:18,016 MigrationManager.java (line 328) 
> Gossiping my schema version f5270873-ba1f-39c7-ab2e-a86db868b09b
> The node2  keeps submitting the migration task over 100+ times to the other 
> node.
> INFO [GossipStage:1] 2016-04-19 11:18:18,261 Gossiper.java (line 1011) Node 
> /192.168.88.33 has restarted, now UP
> INFO [GossipStage:1] 2016-04-19 11:18:18,262 TokenMetadata.java (line 414) 
> Updating topology for /192.168.88.33
> ...
> DEBUG [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 
> 102) Submitting migration task for /192.168.88.33
> ... ( over 100+ times)
> --
> On the otherhand, Node 1 keeps updating its gossip information, followed by 
> receiving and submitting migrationTask afterwards: 
> INFO [RequestResponseStage:3] 2016-04-19 11:18:18,333 Gossiper.java (line 
> 978) InetAddress /192.168.88.34 is now UP
> ...
> DEBUG [MigrationStage:1] 2016-04-19 11:18:18,496 
> MigrationRequestVerbHandler.java (line 41) Received migration request from 
> /192.168.88.34.
> …… ( over 100+ times)
> DEBUG [OptionalTasks:1] 2016-04-19 11:19:18,337 MigrationManager.java (line 
> 127) submitting migration task for /192.168.88.34
> .  (over 50+ times)
> On the side note, we have over 200+ column families defined in Cassandra 
> database, which may related to this amount of rpc traffic.
> P.S.2 The over requested schema migration task will eventually have 
> InternalResponseStage performing schema merge operation. Since this operation 
> requires a compaction for each merge and is much slower to consume. Thus, the 
> back-pressure of incoming schema migration content objects consumes all of 
> the heap space and ultimately ends up OOM!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13784) mismatched input 'default' expecting for CREATE TABLE

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang resolved CASSANDRA-13784.
--
Resolution: Not A Problem

{{default}} was added as keyword in CASSANDRA-11424. Close it as not a problem

> mismatched input 'default' expecting for CREATE TABLE
> -
>
> Key: CASSANDRA-13784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13784
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: {code}
> [root@localhost config]# cat /etc/os-release
> NAME="CentOS Linux"
> VERSION="7 (Core)"
> ID="centos"
> ID_LIKE="rhel fedora"
> VERSION_ID="7"
> PRETTY_NAME="CentOS Linux 7 (Core)"
> ANSI_COLOR="0;31"
> CPE_NAME="cpe:/o:centos:centos:7"
> HOME_URL="https://www.centos.org/;
> BUG_REPORT_URL="https://bugs.centos.org/;
> CENTOS_MANTISBT_PROJECT="CentOS-7"
> CENTOS_MANTISBT_PROJECT_VERSION="7"
> REDHAT_SUPPORT_PRODUCT="centos"
> REDHAT_SUPPORT_PRODUCT_VERSION="7"
> [root@localhost config]# rpm -qa | grep cassandra
> cassandra-3.11.0-1.noarch
> [root@localhost config]# cassandra -v
> 3.11.0
> {code}
>Reporter: timur
>
> Got following error for table create if column name is "default" on 3.11. If 
> I change column name for  example to "defaults" table successfully creates.
> For 3.9 version table with "default" column name successfully creates.  
> {code:sql}
> cqlsh:api_production> CREATE TABLE tasks (id uuid PRIMARY KEY, default 
> boolean, name text, tenant_id uuid) ;
> SyntaxException: line 1:41 mismatched input 'default' expecting ')' (...(id 
> uuid PRIMARY KEY, [default]...)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13795) DDL statements running slow on huge data

[Vladimir (JIRA)]

Vladimir created CASSANDRA-13795:


 Summary: DDL statements running slow on huge data
 Key: CASSANDRA-13795
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13795
 Project: Cassandra
  Issue Type: Bug
Reporter: Vladimir
 Fix For: 3.0.x


We are facing the issues with Cassandra DDL statements with a huge amount of 
data inside. I run statement for create type and it works for several hours 
without success in cqlsh for remote server.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13747) Fix AssertionError in short read protection

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-13747:
--
Status: Patch Available  (was: In Progress)

Thanks for the review.

I've decided to tackle the related issue you in a [separate 
ticket|https://issues.apache.org/jira/browse/CASSANDRA-13794].

Will commit this once tests are happy.

> Fix AssertionError in short read protection
> ---
>
> Key: CASSANDRA-13747
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13747
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x
>
>
> {{ShortReadRowProtection.moreContents()}} expects that by the time we get to 
> that method, the global post-reconciliation counter was already applied to 
> the current partition. However, sometimes it won’t get applied, and the 
> global counter continues counting with {{rowInCurrentPartition}} value not 
> reset from previous partition, which in the most obvious case would trigger 
> the assertion we are observing - {{assert 
> !postReconciliationCounter.isDoneForPartition();}}. In other cases it’s 
> possible because of this lack of reset to query a node for too few extra 
> rows, causing unnecessary SRP data requests.
> Why is the counter not always applied to the current partition?
> The merged {{PartitionIterator}} returned from {{DataResolver.resolve()}} has 
> two transformations applied to it, in the following order:
> {{Filter}} - to purge non-live data from partitions, and to discard empty 
> partitions altogether (except for Thrift)
> {{Counter}}, to count and stop iteration
> Problem is, {{Filter}} ’s {{applyToPartition()}} code that discards empty 
> partitions ({{closeIfEmpty()}} method) would sometimes consume the iterator, 
> triggering short read protection *before* {{Counter}} ’s 
> {{applyToPartition()}} gets called and resets its {{rowInCurrentPartition}} 
> sub-counter.
> We should not be consuming iterators until all transformations are applied to 
> them. For transformations it means that they cannot consume iterators unless 
> they are the last transformation on the stack.
> The linked branch fixes the problem by splitting {{Filter}} into two 
> transformations. The original - {{Filter}} - that does filtering within 
> partitions - and a separate {{EmptyPartitionsDiscarder}}, that discards empty 
> partitions from {{PartitionIterators}}. Thus {{DataResolve.resolve()}}, when 
> constructing its {{PartitionIterator}}, now does merge first, then applies 
> {{Filter}}, then {{Counter}}, and only then, as its last (third) 
> transformation - the {{EmptyPartitionsDiscarder}}. Being the last one 
> applied, it’s legal for it to consume the iterator, and triggering 
> {{moreContents()}} is now no longer a problem.
> Fixes: [3.0|https://github.com/iamaleksey/cassandra/commits/13747-3.0], 
> [3.11|https://github.com/iamaleksey/cassandra/commits/13747-3.11], 
> [4.0|https://github.com/iamaleksey/cassandra/commits/13747-4.0]. dtest 
> [here|https://github.com/iamaleksey/cassandra-dtest/commits/13747].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13794) Fix short read protection logic for querying more rows

[Aleksey Yeschenko (JIRA)]

Aleksey Yeschenko created CASSANDRA-13794:
-

 Summary: Fix short read protection logic for querying more rows
 Key: CASSANDRA-13794
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13794
 Project: Cassandra
  Issue Type: Bug
  Components: Coordination
Reporter: Benedict
Assignee: Aleksey Yeschenko
 Fix For: 3.0.x, 3.11.x


Discovered by [~benedict] while reviewing CASSANDRA-13747:

{quote}
While reviewing I got a little suspicious of the modified line {{DataResolver}} 
:479, as it seemed that n and x were the wrong way around... and, reading the 
comment of intent directly above, and reproducing the calculation, they are 
indeed.

This is probably a significant enough bug that it warrants its own ticket for 
record keeping, though I'm fairly agnostic on that decision.

I'm a little concerned about our current short read behaviour, as right now it 
seems we should be requesting exactly one row, for any size of under-read, 
which could mean extremely poor performance in case of large under-reads.

I would suggest that the outer unconditional {{Math.max}} is a bad idea, has 
been (poorly) insulating us from this error, and that we should first be 
asserting that the calculation yields a value >= 0 before setting to 1.
{quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13747) Fix AssertionError in short read protection

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-13747:
--
Summary: Fix AssertionError in short read protection  (was: Fix short read 
protection)

> Fix AssertionError in short read protection
> ---
>
> Key: CASSANDRA-13747
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13747
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x
>
>
> {{ShortReadRowProtection.moreContents()}} expects that by the time we get to 
> that method, the global post-reconciliation counter was already applied to 
> the current partition. However, sometimes it won’t get applied, and the 
> global counter continues counting with {{rowInCurrentPartition}} value not 
> reset from previous partition, which in the most obvious case would trigger 
> the assertion we are observing - {{assert 
> !postReconciliationCounter.isDoneForPartition();}}. In other cases it’s 
> possible because of this lack of reset to query a node for too few extra 
> rows, causing unnecessary SRP data requests.
> Why is the counter not always applied to the current partition?
> The merged {{PartitionIterator}} returned from {{DataResolver.resolve()}} has 
> two transformations applied to it, in the following order:
> {{Filter}} - to purge non-live data from partitions, and to discard empty 
> partitions altogether (except for Thrift)
> {{Counter}}, to count and stop iteration
> Problem is, {{Filter}} ’s {{applyToPartition()}} code that discards empty 
> partitions ({{closeIfEmpty()}} method) would sometimes consume the iterator, 
> triggering short read protection *before* {{Counter}} ’s 
> {{applyToPartition()}} gets called and resets its {{rowInCurrentPartition}} 
> sub-counter.
> We should not be consuming iterators until all transformations are applied to 
> them. For transformations it means that they cannot consume iterators unless 
> they are the last transformation on the stack.
> The linked branch fixes the problem by splitting {{Filter}} into two 
> transformations. The original - {{Filter}} - that does filtering within 
> partitions - and a separate {{EmptyPartitionsDiscarder}}, that discards empty 
> partitions from {{PartitionIterators}}. Thus {{DataResolve.resolve()}}, when 
> constructing its {{PartitionIterator}}, now does merge first, then applies 
> {{Filter}}, then {{Counter}}, and only then, as its last (third) 
> transformation - the {{EmptyPartitionsDiscarder}}. Being the last one 
> applied, it’s legal for it to consume the iterator, and triggering 
> {{moreContents()}} is now no longer a problem.
> Fixes: [3.0|https://github.com/iamaleksey/cassandra/commits/13747-3.0], 
> [3.11|https://github.com/iamaleksey/cassandra/commits/13747-3.11], 
> [4.0|https://github.com/iamaleksey/cassandra/commits/13747-4.0]. dtest 
> [here|https://github.com/iamaleksey/cassandra-dtest/commits/13747].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139961#comment-16139961
 ] 

Joshua McKenzie commented on CASSANDRA-11713:
-

[~pauloricardomg]: Tidying up backlog - what's status on this? Rebase and 
review?
I'm happy to take on the rebase since it's been lingering on account of me not 
having reviewed it for... over a year. =/

> Add ability to log thread dump when NTR pool is blocked
> ---
>
> Key: CASSANDRA-11713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11713
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Minor
> Attachments: ThreadDumper.png
>
>
> Thread dumps are very useful for troubleshooting Native-Transport-Requests 
> contention issues like CASSANDRA-11363 and CASSANDRA-11529.
> While they could be generated externally with {{jstack}}, sometimes the 
> conditions are transient and it's hard to catch the exact moment when they 
> happen, so it could be useful to generate and log them upon user request when 
> certain internal condition happens.
> I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} 
> that when enabled via JMX generates and logs a single thread dump on the 
> system log when the thread pool queue is full.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-9501) ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata failing

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie resolved CASSANDRA-9501.

Resolution: Cannot Reproduce
  Reviewer:   (was: Jonathan Ellis)

[Test is clean 
now|https://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_utest/lastCompletedBuild/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata/history/].
 Closing.

> ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata failing
> 
>
> Key: CASSANDRA-9501
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9501
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>  Labels: test
> Fix For: 2.2.x
>
> Attachments: 9501_v1.txt, 9501_v2.txt
>
>
> {noformat}
> [junit] Testcase: 
> testSliceByNamesCommandOldMetadata(org.apache.cassandra.db.ColumnFamilyStoreTest):
> FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at 
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:171)
> [junit] at 
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> [junit] at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:266)
> [junit] at 
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:766)
> [junit] at 
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata(ColumnFamilyStoreTest.java:1125)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-8498) Replaying commit log records that are older than gc_grace is dangerous

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-8498:
--

Assignee: (was: Joshua McKenzie)

> Replaying commit log records that are older than gc_grace is dangerous
> --
>
> Key: CASSANDRA-8498
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8498
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>
> If we replay commit log records that are older than gc_grace we could 
> introduce data corruption to the cluster. We should either (1) fail and 
> suggest a repair, or (2) log an exception. I prefer (1).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13363) java.lang.ArrayIndexOutOfBoundsException: null

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135088#comment-16135088
 ] 

Aleksey Yeschenko edited comment on CASSANDRA-13363 at 8/24/17 12:07 PM:
-

[~zhaoyan] Thanks again for the initial analysis. And that patch would 100% 
prevent the AIOOBE, just not fix the underlying issue entirely.

A more comprehensive refactoring to fix the race for good:
||3.0||3.11||4.0||
|[branch|https://github.com/iamaleksey/cassandra/tree/13363-3.0]|[branch|https://github.com/iamaleksey/cassandra/tree/13363-3.11]|[branch|https://github.com/iamaleksey/cassandra/tree/13363-4.0]|
|[utest|https://circleci.com/gh/iamaleksey/cassandra/2]|[utest|https://circleci.com/gh/iamaleksey/cassandra/3]|[utest|https://circleci.com/gh/iamaleksey/cassandra/4]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/211/testReport/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/212/testReport/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/213/testReport/]|
[~beobal] can you please review?

EDIT: Added test results


was (Author: iamaleksey):
[~zhaoyan] Thanks again for the initial analysis. And that patch would 100% 
prevent the AIOOBE, just not fix the underlying issue entirely.

A more comprehensive refactoring to fix the race for good: 
[3.0|https://github.com/iamaleksey/cassandra/tree/13363-3.0], 
[3.11|https://github.com/iamaleksey/cassandra/tree/13363-3.11], 
[4.0|https://github.com/iamaleksey/cassandra/tree/13363-4.0]. Not yet ready to 
commit, as waiting for tests results. [~beobal] can you please review?

> java.lang.ArrayIndexOutOfBoundsException: null
> --
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
>  Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
>Priority: Critical
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a 
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13640) CQLSH error when using 'login' to switch users

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139901#comment-16139901
 ] 

ZhaoYang commented on CASSANDRA-13640:
--

Thansk for the patch. +1

> CQLSH error when using 'login' to switch users
> --
>
> Key: CASSANDRA-13640
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13640
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Fix For: 3.0.x
>
>
> Using {{PasswordAuthenticator}} and {{CassandraAuthorizer}}:
> {code}
> bin/cqlsh -u cassandra -p cassandra
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.14-SNAPSHOT | CQL spec 3.4.0 | Native protocol 
> v4]
> Use HELP for help.
> cassandra@cqlsh> create role super with superuser = true and password = 'p' 
> and login = true;
> cassandra@cqlsh> login super;
> Password:
> super@cqlsh> list roles;
> 'Row' object has no attribute 'values'
> {code}
> When we initialize the Shell, we configure certain settings on the session 
> object such as
> {code}
> self.session.default_timeout = request_timeout
> self.session.row_factory = ordered_dict_factory
> self.session.default_consistency_level = cassandra.ConsistencyLevel.ONE
> {code}
> However, once we perform a LOGIN cmd, which calls do_login(..), we create a 
> new cluster/session object but actually never set those settings on the new 
> session.
> It isn't failing on 3.x. 
> As a workaround, it is possible to logout and log back in and things work 
> correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13640) CQLSH error when using 'login' to switch users

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-13640:
-
Status: Ready to Commit  (was: Patch Available)

> CQLSH error when using 'login' to switch users
> --
>
> Key: CASSANDRA-13640
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13640
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Fix For: 3.0.x
>
>
> Using {{PasswordAuthenticator}} and {{CassandraAuthorizer}}:
> {code}
> bin/cqlsh -u cassandra -p cassandra
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.14-SNAPSHOT | CQL spec 3.4.0 | Native protocol 
> v4]
> Use HELP for help.
> cassandra@cqlsh> create role super with superuser = true and password = 'p' 
> and login = true;
> cassandra@cqlsh> login super;
> Password:
> super@cqlsh> list roles;
> 'Row' object has no attribute 'values'
> {code}
> When we initialize the Shell, we configure certain settings on the session 
> object such as
> {code}
> self.session.default_timeout = request_timeout
> self.session.row_factory = ordered_dict_factory
> self.session.default_consistency_level = cassandra.ConsistencyLevel.ONE
> {code}
> However, once we perform a LOGIN cmd, which calls do_login(..), we create a 
> new cluster/session object but actually never set those settings on the new 
> session.
> It isn't failing on 3.x. 
> As a workaround, it is possible to logout and log back in and things work 
> correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13622) Better config validation/documentation

[ZhaoYang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-13622:
-
Reviewer: Kurt Greaves

> Better config validation/documentation
> --
>
> Key: CASSANDRA-13622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13622
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Kurt Greaves
>Assignee: ZhaoYang
>Priority: Minor
>  Labels: lhf
> Fix For: 4.0
>
>
> There are a number of properties in the yaml that are "in_mb", however 
> resolve to bytes when calculated in {{DatabaseDescriptor.java}}, but are 
> stored in int's. This means that their maximum values are 2047, as any higher 
> when converted to bytes overflows the int.
> Where possible/reasonable we should convert these to be long's, and stored as 
> long's. If there is no reason for the value to ever be >2047 we should at 
> least document that as the max value, or better yet make it error if set 
> higher than that. Noting that although it's bad practice to increase a lot of 
> them to such high values, there may be cases where it is necessary and in 
> which case we should handle it appropriately rather than overflowing and 
> surprising the user. That is, causing it to break but not in the way the user 
> expected it to :)
> Following are functions that currently could be at risk of the above:
> {code:java|title=DatabaseDescriptor.java}
> getThriftFramedTransportSize()
> getMaxValueSize()
> getCompactionLargePartitionWarningThreshold()
> getCommitLogSegmentSize()
> getNativeTransportMaxFrameSize()
> # These are in KB so max value of 2096128
> getBatchSizeWarnThreshold()
> getColumnIndexSize()
> getColumnIndexCacheSize()
> getMaxMutationSize()
> {code}
> Note we may not actually need to fix all of these, and there may be more. 
> This was just from a rough scan over the code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13793) Regression in 3.0, breaking the fix from CASSANDRA-6069

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-13793.
--
Resolution: Invalid

Ignore me, I'm a moron, I jumped to conclusion without properly reading the 
code. The code from 3.0 does properly set collection tombstones to {{t - 1}}. 
There does seem to be a small behavior difference around this code between 2.1 
and 3.0 in that 2.1 was using {{t}} for cell tombstones while 3.0 is now using 
{{t - 1}} but that's unrelated to what I wrote above so closing that.

> Regression in 3.0, breaking the fix from CASSANDRA-6069
> ---
>
> Key: CASSANDRA-13793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13793
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> The goal of the fix of CASSANDRA-6069 was to make sure that collection 
> tombstones in an update in CAS were using {{t-1}} because at least in 
> {{INSERT}} collection tombstones are used to delete data prior to the update 
> but shouldn't delete the newly inserted data itself. Because in 2.x the 
> collection tombstones are normal range tombstones and thus part of the 
> {{DeletionInfo}}, we went with the easy solution of using {{t-1}} for all of 
> {{DeletionInfo}}.
> When moving that code to 3.0, this was migrated too literally however and 
> only the {{DeletionInfo}} got the {{t -1}}. But in 3.0, range tombstones are 
> not part of {{DeletionInfo}} anymore, and so this is broken.
> Thanks to [~aweisberg] for noticing this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13793) Regression in 3.0, breaking the fix from CASSANDRA-6069

[Sylvain Lebresne (JIRA)]

Sylvain Lebresne created CASSANDRA-13793:


 Summary: Regression in 3.0, breaking the fix from CASSANDRA-6069
 Key: CASSANDRA-13793
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13793
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 3.0.x, 3.11.x


The goal of the fix of CASSANDRA-6069 was to make sure that collection 
tombstones in an update in CAS were using {{t-1}} because at least in 
{{INSERT}} collection tombstones are used to delete data prior to the update 
but shouldn't delete the newly inserted data itself. Because in 2.x the 
collection tombstones are normal range tombstones and thus part of the 
{{DeletionInfo}}, we went with the easy solution of using {{t-1}} for all of 
{{DeletionInfo}}.

When moving that code to 3.0, this was migrated too literally however and only 
the {{DeletionInfo}} got the {{t -1}}. But in 3.0, range tombstones are not 
part of {{DeletionInfo}} anymore, and so this is broken.

Thanks to [~aweisberg] for noticing this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13719) Potential AssertionError during ReadRepair of range tombstone and partition deletions

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13719:
-
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.11.1
   3.0.15

> Potential AssertionError during ReadRepair of range tombstone and partition 
> deletions
> -
>
> Key: CASSANDRA-13719
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13719
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.15, 3.11.1
>
>
> When reconciling range tombstones for read repair in 
> {{DataResolver.RepairMergeListener.MergeListener}}, when we check if there is 
> ongoing deletion repair for a source, we don't look for partition level 
> deletions which throw off the logic and can throw an AssertionError.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13719) Potential AssertionError during ReadRepair of range tombstone and partition deletions

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13719:
-
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

Alright, committed, thanks.

> Potential AssertionError during ReadRepair of range tombstone and partition 
> deletions
> -
>
> Key: CASSANDRA-13719
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13719
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> When reconciling range tombstones for read repair in 
> {{DataResolver.RepairMergeListener.MergeListener}}, when we check if there is 
> ongoing deletion repair for a source, we don't look for partition level 
> deletions which throw off the logic and can throw an AssertionError.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Potential AssertionError during ReadRepair of range tombstone and partition deletions

[slebresne]

Potential AssertionError during ReadRepair of range tombstone and partition 
deletions

patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13719


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5e57dd14
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5e57dd14
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5e57dd14

Branch: refs/heads/cassandra-3.11
Commit: 5e57dd14eb37adf06d2105227e0105d871ea6f76
Parents: 6a1b1f2
Author: Sylvain Lebresne 
Authored: Fri Jul 21 16:58:53 2017 +0200
Committer: Sylvain Lebresne 
Committed: Thu Aug 24 11:28:57 2017 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 22 +
 .../db/partitions/AbstractBTreePartition.java   | 14 +--
 .../db/partitions/PartitionUpdate.java  |  9 ++
 .../apache/cassandra/service/DataResolver.java  | 70 +--
 .../cassandra/service/DataResolverTest.java | 91 +++-
 6 files changed, 192 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97dda05..2b49bc3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
  * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
  * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/src/java/org/apache/cassandra/db/ReadResponse.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadResponse.java 
b/src/java/org/apache/cassandra/db/ReadResponse.java
index 693b52b..c59d00a 100644
--- a/src/java/org/apache/cassandra/db/ReadResponse.java
+++ b/src/java/org/apache/cassandra/db/ReadResponse.java
@@ -92,6 +92,28 @@ public abstract class ReadResponse
 
 public abstract boolean isDigestResponse();
 
+/**
+ * Creates a string of the requested partition in this read response 
suitable for debugging.
+ */
+public String toDebugString(ReadCommand command, DecoratedKey key)
+{
+if (isDigestResponse())
+return "Digest:0x" + ByteBufferUtil.bytesToHex(digest(command));
+
+try (UnfilteredPartitionIterator iter = makeIterator(command))
+{
+while (iter.hasNext())
+{
+try (UnfilteredRowIterator partition = iter.next())
+{
+if (partition.partitionKey().equals(key))
+return 
ImmutableBTreePartition.create(partition).toString();
+}
+}
+}
+return "";
+}
+
 protected static ByteBuffer makeDigest(UnfilteredPartitionIterator 
iterator, ReadCommand command)
 {
 MessageDigest digest = FBUtilities.threadLocalMD5Digest();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--
diff --git 
a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java 
b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
index 2aa622e..1f3dbd0 100644
--- a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
+++ b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
@@ -99,7 +99,7 @@ public abstract class AbstractBTreePartition implements 
Partition, Iterable
 
 public DeletionTime partitionLevelDeletion()
 {
-return holder().deletionInfo.getPartitionDeletion();
+return deletionInfo().getPartitionDeletion();
 }
 
 public PartitionColumns columns()
@@ -372,17 +372,21 @@ public abstract class AbstractBTreePartition implements 
Partition, Iterable
 {
 StringBuilder sb = new StringBuilder();
 
-sb.append(String.format("[%s.%s] key=%s columns=%s",
+sb.append(String.format("[%s.%s] key=%s partition_deletion=%s 
columns=%s",
 metadata.ksName,
 metadata.cfName,
 
metadata.getKeyValidator().getString(partitionKey().getKey()),
+partitionLevelDeletion(),
 columns()));
 
 if (staticRow() != Rows.EMPTY_STATIC_ROW)
-sb.append("\n").append(staticRow().toString(metadata));
+ 

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.11

* cassandra-3.0:
  Potential AssertionError during ReadRepair of range tombstone and partition 
deletions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed9b04d6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed9b04d6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed9b04d6

Branch: refs/heads/trunk
Commit: ed9b04d6aeffc85a2dbcd8c136df769491f79d85
Parents: 59d4c27 5e57dd1
Author: Sylvain Lebresne 
Authored: Thu Aug 24 11:36:14 2017 +0200
Committer: Sylvain Lebresne 
Committed: Thu Aug 24 11:36:14 2017 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 22 +
 .../db/partitions/AbstractBTreePartition.java   | 14 +--
 .../db/partitions/PartitionUpdate.java  |  9 ++
 .../apache/cassandra/service/DataResolver.java  | 70 +--
 .../cassandra/service/DataResolverTest.java | 91 +++-
 6 files changed, 192 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/CHANGES.txt
--
diff --cc CHANGES.txt
index 9e42ffb,2b49bc3..b22cb5e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
 -3.0.15
 +3.11.1
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
   * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
   * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
   * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/db/ReadResponse.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
--
diff --cc src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
index 64d7dd1,7bd5345..435c324
--- a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
+++ b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
@@@ -314,9 -296,18 +314,18 @@@ public class PartitionUpdate extends Ab
  
  int nowInSecs = FBUtilities.nowInSeconds();
  List asIterators = Lists.transform(updates, 
AbstractBTreePartition::unfilteredIterator);
 -return fromIterator(UnfilteredRowIterators.merge(asIterators, 
nowInSecs));
 +return fromIterator(UnfilteredRowIterators.merge(asIterators, 
nowInSecs), ColumnFilter.all(updates.get(0).metadata()));
  }
  
+ // We override this, because the version in the super-class calls 
holder(), which build the update preventing
+ // further updates, but that's not necessary here and being able to check 
at least the partition deletion without
+ // "locking" the update is nice (and used in 
DataResolver.RepairMergeListener.MergeListener).
+ @Override
+ public DeletionInfo deletionInfo()
+ {
+ return deletionInfo;
+ }
+ 
  /**
   * Modify this update to set every timestamp for live data to {@code 
newTimestamp} and
   * every deletion timestamp to {@code newTimestamp - 1}.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/service/DataResolver.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/test/unit/org/apache/cassandra/service/DataResolverTest.java
--
diff --cc test/unit/org/apache/cassandra/service/DataResolverTest.java
index 8e4f385,65e18ce..2b1e095
--- a/test/unit/org/apache/cassandra/service/DataResolverTest.java
+++ b/test/unit/org/apache/cassandra/service/DataResolverTest.java
@@@ -576,9 -574,9 +576,9 @@@ public 

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[slebresne]

Merge branch 'cassandra-3.11' into trunk

* cassandra-3.11:
  Potential AssertionError during ReadRepair of range tombstone and partition 
deletions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/652d9f64
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/652d9f64
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/652d9f64

Branch: refs/heads/trunk
Commit: 652d9f64f14d8375a8412561271a7abf27722f20
Parents: d2dcd7f ed9b04d
Author: Sylvain Lebresne 
Authored: Thu Aug 24 11:39:35 2017 +0200
Committer: Sylvain Lebresne 
Committed: Thu Aug 24 11:39:35 2017 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 22 +
 .../db/partitions/AbstractBTreePartition.java   | 16 ++--
 .../db/partitions/PartitionUpdate.java  |  9 ++
 .../apache/cassandra/service/DataResolver.java  | 70 +--
 .../cassandra/service/DataResolverTest.java | 91 +++-
 6 files changed, 193 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/652d9f64/CHANGES.txt
--
diff --cc CHANGES.txt
index d0ec78d,b22cb5e..e19e9dd
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -129,7 -6,9 +129,8 @@@
   * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
   * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
  Merged from 3.0:
+  * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
   * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
 - * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
   * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)
   * Fix digest calculation for counter cells (CASSANDRA-13750)
   * Fix ColumnDefinition.cellValueType() for non-frozen collection and change 
SSTabledump to use type.toJSONString() (CASSANDRA-13573)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/652d9f64/src/java/org/apache/cassandra/db/ReadResponse.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/652d9f64/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--
diff --cc 
src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
index d8f2856,90db062..d913cb3
--- a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
+++ b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
@@@ -94,10 -99,10 +94,10 @@@ public abstract class AbstractBTreePart
  
  public DeletionTime partitionLevelDeletion()
  {
- return holder().deletionInfo.getPartitionDeletion();
+ return deletionInfo().getPartitionDeletion();
  }
  
 -public PartitionColumns columns()
 +public RegularAndStaticColumns columns()
  {
  return holder().columns;
  }
@@@ -317,16 -322,21 +317,20 @@@
  {
  StringBuilder sb = new StringBuilder();
  
- sb.append(String.format("[%s] key=%s columns=%s",
- metadata().toString(),
 -sb.append(String.format("[%s.%s] key=%s partition_deletion=%s 
columns=%s",
 -metadata.ksName,
 -metadata.cfName,
 -
metadata.getKeyValidator().getString(partitionKey().getKey()),
++sb.append(String.format("[%s] key=%s partition_deletion=%s 
columns=%s",
++metadata(),
 +
metadata().partitionKeyType.getString(partitionKey().getKey()),
+ partitionLevelDeletion(),
  columns()));
  
  if (staticRow() != Rows.EMPTY_STATIC_ROW)
- sb.append("\n").append(staticRow().toString(metadata()));
 -sb.append("\n").append(staticRow().toString(metadata, true));
++sb.append("\n").append(staticRow().toString(metadata(), 
true));
  
- for (Row row : this)
- sb.append("\n").append(row.toString(metadata()));
+ try (UnfilteredRowIterator iter = unfilteredIterator())
+ {
+ while (iter.hasNext())
 -sb.append("\n").append(iter.next().toString(metadata, 
true));
++sb.append("\n").append(iter.next().toString(metadata(), 
true));
+ }
  
  return sb.toString();
  }

[1/6] cassandra git commit: Potential AssertionError during ReadRepair of range tombstone and partition deletions

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 6a1b1f26b -> 5e57dd14e
  refs/heads/cassandra-3.11 59d4c2719 -> ed9b04d6a
  refs/heads/trunk d2dcd7f88 -> 652d9f64f


Potential AssertionError during ReadRepair of range tombstone and partition 
deletions

patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13719


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5e57dd14
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5e57dd14
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5e57dd14

Branch: refs/heads/cassandra-3.0
Commit: 5e57dd14eb37adf06d2105227e0105d871ea6f76
Parents: 6a1b1f2
Author: Sylvain Lebresne 
Authored: Fri Jul 21 16:58:53 2017 +0200
Committer: Sylvain Lebresne 
Committed: Thu Aug 24 11:28:57 2017 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 22 +
 .../db/partitions/AbstractBTreePartition.java   | 14 +--
 .../db/partitions/PartitionUpdate.java  |  9 ++
 .../apache/cassandra/service/DataResolver.java  | 70 +--
 .../cassandra/service/DataResolverTest.java | 91 +++-
 6 files changed, 192 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97dda05..2b49bc3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
  * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
  * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/src/java/org/apache/cassandra/db/ReadResponse.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadResponse.java 
b/src/java/org/apache/cassandra/db/ReadResponse.java
index 693b52b..c59d00a 100644
--- a/src/java/org/apache/cassandra/db/ReadResponse.java
+++ b/src/java/org/apache/cassandra/db/ReadResponse.java
@@ -92,6 +92,28 @@ public abstract class ReadResponse
 
 public abstract boolean isDigestResponse();
 
+/**
+ * Creates a string of the requested partition in this read response 
suitable for debugging.
+ */
+public String toDebugString(ReadCommand command, DecoratedKey key)
+{
+if (isDigestResponse())
+return "Digest:0x" + ByteBufferUtil.bytesToHex(digest(command));
+
+try (UnfilteredPartitionIterator iter = makeIterator(command))
+{
+while (iter.hasNext())
+{
+try (UnfilteredRowIterator partition = iter.next())
+{
+if (partition.partitionKey().equals(key))
+return 
ImmutableBTreePartition.create(partition).toString();
+}
+}
+}
+return "";
+}
+
 protected static ByteBuffer makeDigest(UnfilteredPartitionIterator 
iterator, ReadCommand command)
 {
 MessageDigest digest = FBUtilities.threadLocalMD5Digest();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--
diff --git 
a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java 
b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
index 2aa622e..1f3dbd0 100644
--- a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
+++ b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
@@ -99,7 +99,7 @@ public abstract class AbstractBTreePartition implements 
Partition, Iterable
 
 public DeletionTime partitionLevelDeletion()
 {
-return holder().deletionInfo.getPartitionDeletion();
+return deletionInfo().getPartitionDeletion();
 }
 
 public PartitionColumns columns()
@@ -372,17 +372,21 @@ public abstract class AbstractBTreePartition implements 
Partition, Iterable
 {
 StringBuilder sb = new StringBuilder();
 
-sb.append(String.format("[%s.%s] key=%s columns=%s",
+sb.append(String.format("[%s.%s] key=%s partition_deletion=%s 
columns=%s",
 metadata.ksName,
 metadata.cfName,
 
metadata.getKeyValidator().getString(partitionKey().getKey()),
+

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.11

* cassandra-3.0:
  Potential AssertionError during ReadRepair of range tombstone and partition 
deletions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed9b04d6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed9b04d6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed9b04d6

Branch: refs/heads/cassandra-3.11
Commit: ed9b04d6aeffc85a2dbcd8c136df769491f79d85
Parents: 59d4c27 5e57dd1
Author: Sylvain Lebresne 
Authored: Thu Aug 24 11:36:14 2017 +0200
Committer: Sylvain Lebresne 
Committed: Thu Aug 24 11:36:14 2017 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 22 +
 .../db/partitions/AbstractBTreePartition.java   | 14 +--
 .../db/partitions/PartitionUpdate.java  |  9 ++
 .../apache/cassandra/service/DataResolver.java  | 70 +--
 .../cassandra/service/DataResolverTest.java | 91 +++-
 6 files changed, 192 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/CHANGES.txt
--
diff --cc CHANGES.txt
index 9e42ffb,2b49bc3..b22cb5e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
 -3.0.15
 +3.11.1
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Potential AssertionError during ReadRepair of range tombstone and 
partition deletions (CASSANDRA-13719)
   * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
   * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
   * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/db/ReadResponse.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
--
diff --cc src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
index 64d7dd1,7bd5345..435c324
--- a/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
+++ b/src/java/org/apache/cassandra/db/partitions/PartitionUpdate.java
@@@ -314,9 -296,18 +314,18 @@@ public class PartitionUpdate extends Ab
  
  int nowInSecs = FBUtilities.nowInSeconds();
  List asIterators = Lists.transform(updates, 
AbstractBTreePartition::unfilteredIterator);
 -return fromIterator(UnfilteredRowIterators.merge(asIterators, 
nowInSecs));
 +return fromIterator(UnfilteredRowIterators.merge(asIterators, 
nowInSecs), ColumnFilter.all(updates.get(0).metadata()));
  }
  
+ // We override this, because the version in the super-class calls 
holder(), which build the update preventing
+ // further updates, but that's not necessary here and being able to check 
at least the partition deletion without
+ // "locking" the update is nice (and used in 
DataResolver.RepairMergeListener.MergeListener).
+ @Override
+ public DeletionInfo deletionInfo()
+ {
+ return deletionInfo;
+ }
+ 
  /**
   * Modify this update to set every timestamp for live data to {@code 
newTimestamp} and
   * every deletion timestamp to {@code newTimestamp - 1}.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/src/java/org/apache/cassandra/service/DataResolver.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed9b04d6/test/unit/org/apache/cassandra/service/DataResolverTest.java
--
diff --cc test/unit/org/apache/cassandra/service/DataResolverTest.java
index 8e4f385,65e18ce..2b1e095
--- a/test/unit/org/apache/cassandra/service/DataResolverTest.java
+++ b/test/unit/org/apache/cassandra/service/DataResolverTest.java
@@@ -576,9 -574,9 +576,9 

[3/6] cassandra git commit: Potential AssertionError during ReadRepair of range tombstone and partition deletions

[slebresne]

Potential AssertionError during ReadRepair of range tombstone and partition 
deletions

patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13719


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5e57dd14
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5e57dd14
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5e57dd14

Branch: refs/heads/trunk
Commit: 5e57dd14eb37adf06d2105227e0105d871ea6f76
Parents: 6a1b1f2
Author: Sylvain Lebresne 
Authored: Fri Jul 21 16:58:53 2017 +0200
Committer: Sylvain Lebresne 
Committed: Thu Aug 24 11:28:57 2017 +0200

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/ReadResponse.java   | 22 +
 .../db/partitions/AbstractBTreePartition.java   | 14 +--
 .../db/partitions/PartitionUpdate.java  |  9 ++
 .../apache/cassandra/service/DataResolver.java  | 70 +--
 .../cassandra/service/DataResolverTest.java | 91 +++-
 6 files changed, 192 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97dda05..2b49bc3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Potential AssertionError during ReadRepair of range tombstone and partition 
deletions (CASSANDRA-13719)
  * Don't let stress write warmup data if n=0 (CASSANDRA-13773)
  * Gossip thread slows down when using batch commit log (CASSANDRA-12966)
  * Randomize batchlog endpoint selection with only 1 or 2 racks 
(CASSANDRA-12884)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/src/java/org/apache/cassandra/db/ReadResponse.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadResponse.java 
b/src/java/org/apache/cassandra/db/ReadResponse.java
index 693b52b..c59d00a 100644
--- a/src/java/org/apache/cassandra/db/ReadResponse.java
+++ b/src/java/org/apache/cassandra/db/ReadResponse.java
@@ -92,6 +92,28 @@ public abstract class ReadResponse
 
 public abstract boolean isDigestResponse();
 
+/**
+ * Creates a string of the requested partition in this read response 
suitable for debugging.
+ */
+public String toDebugString(ReadCommand command, DecoratedKey key)
+{
+if (isDigestResponse())
+return "Digest:0x" + ByteBufferUtil.bytesToHex(digest(command));
+
+try (UnfilteredPartitionIterator iter = makeIterator(command))
+{
+while (iter.hasNext())
+{
+try (UnfilteredRowIterator partition = iter.next())
+{
+if (partition.partitionKey().equals(key))
+return 
ImmutableBTreePartition.create(partition).toString();
+}
+}
+}
+return "";
+}
+
 protected static ByteBuffer makeDigest(UnfilteredPartitionIterator 
iterator, ReadCommand command)
 {
 MessageDigest digest = FBUtilities.threadLocalMD5Digest();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5e57dd14/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
--
diff --git 
a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java 
b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
index 2aa622e..1f3dbd0 100644
--- a/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
+++ b/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java
@@ -99,7 +99,7 @@ public abstract class AbstractBTreePartition implements 
Partition, Iterable
 
 public DeletionTime partitionLevelDeletion()
 {
-return holder().deletionInfo.getPartitionDeletion();
+return deletionInfo().getPartitionDeletion();
 }
 
 public PartitionColumns columns()
@@ -372,17 +372,21 @@ public abstract class AbstractBTreePartition implements 
Partition, Iterable
 {
 StringBuilder sb = new StringBuilder();
 
-sb.append(String.format("[%s.%s] key=%s columns=%s",
+sb.append(String.format("[%s.%s] key=%s partition_deletion=%s 
columns=%s",
 metadata.ksName,
 metadata.cfName,
 
metadata.getKeyValidator().getString(partitionKey().getKey()),
+partitionLevelDeletion(),
 columns()));
 
 if (staticRow() != Rows.EMPTY_STATIC_ROW)
-sb.append("\n").append(staticRow().toString(metadata));
+  

[jira] [Updated] (CASSANDRA-13069) Local batchlog for MV may not be correctly written on node movements

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-13069:

Status: Patch Available  (was: Open)

> Local batchlog for MV may not be correctly written on node movements
> 
>
> Key: CASSANDRA-13069
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13069
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: Sylvain Lebresne
>Assignee: Paulo Motta
>
> Unless I'm really reading this wrong, I think the code 
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L829-L843],
>  which comes from CASSANDRA-10674, isn't working properly.
> More precisely, I believe we can have both paired and unpaired mutations, so 
> that both {{if}} can be taken, but if that's the case, the 2nd write to the 
> batchlog will basically overwrite (remove) the batchlog write of the 1st 
> {{if}} and I don't think that's the intention. In practice, this means 
> "paired" mutation won't be in the batchlog, which mean they won't be replayed 
> at all if they fail.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org