date:20200513



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15806:

Status: Ready to Commit  (was: Review In Progress)

> C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
> 
>
> Key: CASSANDRA-15806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: sequoyha pelletier
>Assignee: sequoyha pelletier
>Priority: Normal
> Fix For: 4.0
>
> Attachments: 15806-4.0.txt
>
>
> Currently, there is no way to identify SSTables that were created as 
> transient replicated data. Even thought the feature is experimental we should 
> open that up for those that want to experiment. This seems to be an 
> oversight. I have added the missing line of code to the SStableMetadataViewer 
> and will attach a patch shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15806:

Fix Version/s: (was: 4.0)
   4.0-alpha

> C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
> 
>
> Key: CASSANDRA-15806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: sequoyha pelletier
>Assignee: sequoyha pelletier
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15806-4.0.txt
>
>
> Currently, there is no way to identify SSTables that were created as 
> transient replicated data. Even thought the feature is experimental we should 
> open that up for those that want to experiment. This seems to be an 
> oversight. I have added the missing line of code to the SStableMetadataViewer 
> and will attach a patch shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106742#comment-17106742
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15806 at 5/14/20, 12:04 AM:


LGTM 
- Changed the field name from Is Transient to IsTransient, considering the 
format of the rest of them in the list.
- CHANGES.txt updated

[Branch | https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15806]
 
I think running the unit tests and inJVM for this one is enough. They were 
green here:
[Java 8 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/695]
[Java 11 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/707]
[Java 8 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/694]
[Java 11 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/693]

[~brandon.williams], do you want to check and commit? (not a committer yet)


was (Author: e.dimitrova):
LGTM Just changed the field name from Is Transient to IsTransient, considering 
the format of the rest of them in the list. 
I think running the unit tests and inJVM for this one is enough. They were 
green here:
[Java 8 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/695]
[Java 11 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/707]
[Java 8 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/694]
[Java 11 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/693]

[~brandon.williams], do you want to check and commit? (not a committer yet)

> C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
> 
>
> Key: CASSANDRA-15806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: sequoyha pelletier
>Assignee: sequoyha pelletier
>Priority: Normal
> Fix For: 4.0
>
> Attachments: 15806-4.0.txt
>
>
> Currently, there is no way to identify SSTables that were created as 
> transient replicated data. Even thought the feature is experimental we should 
> open that up for those that want to experiment. This seems to be an 
> oversight. I have added the missing line of code to the SStableMetadataViewer 
> and will attach a patch shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106742#comment-17106742
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15806 at 5/14/20, 12:00 AM:


LGTM Just changed the field name from Is Transient to IsTransient, considering 
the format of the rest of them in the list. 
I think running the unit tests and inJVM for this one is enough. They were 
green here:
[Java 8 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/695]
[Java 11 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/707]
[Java 8 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/694]
[Java 11 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/693]

[~brandon.williams], do you want to check and commit? (not a committer yet)


was (Author: e.dimitrova):
LGTM Just changed the field name from Is Transient to IsTransient, considering 
the format of the rest of them in the list. 
I think running the unit tests and inJVM for this one is enough. They were 
green here:
[Java 8 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/695]
[Java 11 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/707]
[Java 8 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/694]
[Java 11 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/693]

> C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
> 
>
> Key: CASSANDRA-15806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: sequoyha pelletier
>Assignee: sequoyha pelletier
>Priority: Normal
> Fix For: 4.0
>
> Attachments: 15806-4.0.txt
>
>
> Currently, there is no way to identify SSTables that were created as 
> transient replicated data. Even thought the feature is experimental we should 
> open that up for those that want to experiment. This seems to be an 
> oversight. I have added the missing line of code to the SStableMetadataViewer 
> and will attach a patch shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106742#comment-17106742
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15806:
-

LGTM Just changed the field name from Is Transient to IsTransient, considering 
the format of the rest of them in the list. 
I think running the unit tests and inJVM for this one is enough. They were 
green here:
[Java 8 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/695]
[Java 11 unit tests | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/707]
[Java 8 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/254a32b9-8011-4137-8c4f-605fcf4f7593/jobs/694]
[Java 11 inJVM | 
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/131/workflows/6ab777ee-8038-499e-903f-5fcbf9c00786/jobs/693]

> C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
> 
>
> Key: CASSANDRA-15806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: sequoyha pelletier
>Assignee: sequoyha pelletier
>Priority: Normal
> Fix For: 4.0
>
> Attachments: 15806-4.0.txt
>
>
> Currently, there is no way to identify SSTables that were created as 
> transient replicated data. Even thought the feature is experimental we should 
> open that up for those that want to experiment. This seems to be an 
> oversight. I have added the missing line of code to the SStableMetadataViewer 
> and will attach a patch shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRASC-23) Set up structure for handling multiple Cassandra versions



[ 
https://issues.apache.org/jira/browse/CASSANDRASC-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106639#comment-17106639
 ] 

Jon Haddad commented on CASSANDRASC-23:
---

I went ahead and removed the hard coded references to the 4.0 code from the 
{{CassandraTestTemplate}} to a standalone class, {{TestVersionSupplier}}.

I've also cleaned up the patch a little, and fixed up comments, and improved 
the developer documentation.

> Set up structure for handling multiple Cassandra versions
> -
>
> Key: CASSANDRASC-23
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-23
> Project: Sidecar for Apache Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>Priority: Normal
>
> The first sidecar release will be for Cassandra 4.0, but one of the project 
> goals is to be able to handle multiple versions.  This will be especially 
> important in mixed version clusters, or even mixed version 1:N sidecar to C* 
> nodes (see CASSANDRASC-17).
> This JIRA is to lay the foundational work for supporting multiple Cassandra 
> versions. 
> Each Cassandra version (for example \{{cassandra40}}, \{{cassandra50}}, will 
> be in a separate Gradle subproject, implementing a common interface defined 
> in a \{{common}} subproject.  A common cassandra integration testing 
> framework will be able to test every version to ensure each version adheres 
> to the expected interface.
> Each module will define the minimum compatibility it supports, so if someone 
> (for example) wants to implement a Cassandra30 module for their own cluster, 
> they will have the ability to do so.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15799) CorruptSSTableException when compacting a 3.0 format sstable that was originally created as an outcome of 2.1 sstable upgrade



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106626#comment-17106626
 ] 

David Capwell commented on CASSANDRA-15799:
---

Looking closer I see that CASSANDRA-15373  blocked the data, it didn't reach my 
patch yet.  Thats good as that means CASSANDRA-15373 detects this case so 
3.0.20+ would not get corrupted.

> CorruptSSTableException when compacting a 3.0 format sstable that was 
> originally created as an outcome of 2.1 sstable upgrade
> -
>
> Key: CASSANDRA-15799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15799
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is reproduce-able.
> {code:java}
> DEBUG [CompactionExecutor:10] 2020-05-07 19:33:34,268 CompactionTask.java:158 
> - Compacting (a3ea9fc0-9099-11ea-933f-c5e852f71338) 
> [/mnt/data/cassandra/data/ks/cf/md-10802-big-Data.db:level=0, ]
> ERROR [CompactionExecutor:10] 2020-05-07 19:33:34,275 
> CassandraDaemon.java:208 - Exception in thread 
> Thread[CompactionExecutor:10,1,RMI Runtime]
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/data/cassandra/data/ks/cf/md-10802-big-Data.db
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:105)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.MergeIterator$TrivialOneToOne.computeNext(MergeIterator.java:460)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:534)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:394)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:165)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
>

[jira] [Commented] (CASSANDRA-15799) CorruptSSTableException when compacting a 3.0 format sstable that was originally created as an outcome of 2.1 sstable upgrade



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106611#comment-17106611
 ] 

David Capwell commented on CASSANDRA-15799:
---

Thanks. Dumping context here.

What we see is that the Compact Storage for this table has: Cell.name == 
clusterKey, Cell.value == value field.  We see that Cell.value has calls to 
validate the type in 3.0.20, but there isn't anything for validating the 
Cell.name and clustering.  Based off the data the Cell.name is expected to be a 
bigint, but it has more bytes than expected; which eventually corrupts 
unfiltered serialization.

I will cleanup the POC to guard against this, but the open issue is "why is the 
clustering key not a bigint and how do we recover"

> CorruptSSTableException when compacting a 3.0 format sstable that was 
> originally created as an outcome of 2.1 sstable upgrade
> -
>
> Key: CASSANDRA-15799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15799
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is reproduce-able.
> {code:java}
> DEBUG [CompactionExecutor:10] 2020-05-07 19:33:34,268 CompactionTask.java:158 
> - Compacting (a3ea9fc0-9099-11ea-933f-c5e852f71338) 
> [/mnt/data/cassandra/data/ks/cf/md-10802-big-Data.db:level=0, ]
> ERROR [CompactionExecutor:10] 2020-05-07 19:33:34,275 
> CassandraDaemon.java:208 - Exception in thread 
> Thread[CompactionExecutor:10,1,RMI Runtime]
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/data/cassandra/data/ks/cf/md-10802-big-Data.db
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:105)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.MergeIterator$TrivialOneToOne.computeNext(MergeIterator.java:460)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:534)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:394)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:165)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]

[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11

2020-05-13 Thread Jordan West (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan West updated CASSANDRA-15809:

Change Category: Quality Assurance
 Complexity: Normal
 Status: Open  (was: Triage Needed)

> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> The JDK is specified in the groovy DSL:
> [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
>  
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]
>  
> Background 
> thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15807) Support multiple keyspaces in Cassandra-Diff

2020-05-13 Thread Jordan West (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jordan West updated CASSANDRA-15807:

Change Category: Semantic
 Complexity: Normal
 Status: Open  (was: Triage Needed)

> Support multiple keyspaces in Cassandra-Diff
> 
>
> Key: CASSANDRA-15807
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15807
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tool/diff
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> Adding the support to run diff comparison of tables across multiple keyspaces 
> to avoid bringing up multiple spark jobs and give a better view of data diffs 
> in the whole cluster. 
> Cassandra-diff currently only support compare multiple tables under one 
> single keyspace.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15809) ASF CI builds for JDK11



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106595#comment-17106595
 ] 

David Capwell commented on CASSANDRA-15809:
---

1) java 11 tends to be more flakey than java 8, so this would be very good to 
help show that
2) JDK 14 would be awesome to add, though I believe JDK 13 removed CMS, so 
would require changing GC I think.

> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> The JDK is specified in the groovy DSL:
> [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
>  
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]
>  
> Background 
> thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106587#comment-17106587
 ] 

David Capwell commented on CASSANDRA-15776:
---

Here is the latest Circle CI run: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra/282/workflows/532a7fcf-5d3d-4b9f-bcda-d0bfb73d324e.
 GREEN

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15812) Submitting Validation requests can block ANTI_ENTROPY stage

2020-05-13 Thread Sam Tunnicliffe (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-15812:

 Bug Category: Parent values: Degradation(12984)Level 1 values: Resource 
Management(12995)
   Complexity: Normal
  Component/s: Consistency/Repair
Discovered By: User Report
Fix Version/s: 4.0-alpha
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Submitting Validation requests can block ANTI_ENTROPY stage 
> 
>
> Key: CASSANDRA-15812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15812
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-alpha
>
>
>  RepairMessages are handled on Stage.ANTI_ENTROPY, which has a thread pool 
> with core/max capacity of one, ie. we can only process one message at a time. 
>  
> Scheduling validation compactions may however block the stage completely, by 
> blocking on CompactionManager's ValidationExecutor while submitting a new 
> validation compaction, in cases where there are already more validations 
> running than can be executed in parallel.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15812) Submitting Validation requests can block ANTI_ENTROPY stage

2020-05-13 Thread Sam Tunnicliffe (Jira)

Sam Tunnicliffe created CASSANDRA-15812:
---

 Summary: Submitting Validation requests can block ANTI_ENTROPY 
stage 
 Key: CASSANDRA-15812
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15812
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe


 RepairMessages are handled on Stage.ANTI_ENTROPY, which has a thread pool with 
core/max capacity of one, ie. we can only process one message at a time. 
 
Scheduling validation compactions may however block the stage completely, by 
blocking on CompactionManager's ValidationExecutor while submitting a new 
validation compaction, in cases where there are already more validations 
running than can be executed in parallel.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106397#comment-17106397
 ] 

Michael Semb Wever edited comment on CASSANDRA-15776 at 5/13/20, 6:54 PM:
--

I have a few small nits/questions on naming, see PR.

New ASF CI run is 
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/121/pipeline]


was (Author: michaelsembwever):
Has a few small nits/questions on naming, see PR.

New ASF CI run is 
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/121/pipeline]

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106577#comment-17106577
 ] 

David Capwell commented on CASSANDRA-15776:
---

[~mck] fixed all feedback comments; can you review again?

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15806:

Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova  (was: Ekaterina 
Dimitrova)
   Ekaterina Dimitrova, Ekaterina Dimitrova
   Status: Review In Progress  (was: Patch Available)

> C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
> 
>
> Key: CASSANDRA-15806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: sequoyha pelletier
>Assignee: sequoyha pelletier
>Priority: Normal
> Fix For: 4.0
>
> Attachments: 15806-4.0.txt
>
>
> Currently, there is no way to identify SSTables that were created as 
> transient replicated data. Even thought the feature is experimental we should 
> open that up for those that want to experiment. This seems to be an 
> oversight. I have added the missing line of code to the SStableMetadataViewer 
> and will attach a patch shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect

2020-05-13 Thread Benedict Elliott Smith (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106575#comment-17106575
 ] 

Benedict Elliott Smith commented on CASSANDRA-15727:


Sorry that I'm late to this party, but might it be cleaner to move the setting 
of {{messagingVersion}} into {{attempt}} since we already do it there, and 
remove it from {{onCompletedHandshake/RETRY}} and the class constructor?  This 
would capture all possible reasons {{messagingVersion}} might have changed, and 
reduce the verbosity of this particular point in the code, possibly keeping its 
intent a bit clearer.

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15797) Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest

2020-05-13 Thread Yifan Cai (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106541#comment-17106541
 ] 

Yifan Cai commented on CASSANDRA-15797:
---

Thanks [~vinaykumarcse] for reviewing. 

I have updated the PR to address your comments. 

> Fix flaky BinLogTest - org.apache.cassandra.utils.binlog.BinLogTest
> ---
>
> Key: CASSANDRA-15797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15797
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> An internal CI system is failing BinLogTest somewhat frequently under JDK11.  
> Configuration was recently changed to reduce the number of cores the tests 
> run with, however it is reproducible on an 8 core laptop.
> {code}
> [junit-timeout] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC 
> was deprecated in version 9.0 and will likely be removed in a future release.
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [Junit-timeout] WARNING: An illegal reflective access operation has occurred
> [junit-timeout] WARNING: Illegal reflective access by 
> net.openhft.chronicle.core.Jvm (file:/.../lib/chronicle-core-1.16.4.jar) to 
> field java.nio.Bits.RESERVED_MEMORY
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of 
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of 
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a 
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests 
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.895 sec
> [junit-timeout]
> [junit-timeout] Testcase: 
> testPutAfterStop(org.apache.cassandra.utils.binlog.BinLogTest): FAILED
> [junit-timeout] expected: but 
> was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout] at 
> org.apache.cassandra.utils.binlog.BinLogTest.testPutAfterStop(BinLogTest.java:431)
> [junit-timeout] at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> There's also a different failure under JDK8
> {code}
> junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests 
> run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 15.273 sec
> [junit-timeout]
> [junit-timeout] Testcase: 
> testBinLogStartStop(org.apache.cassandra.utils.binlog.BinLogTest):  FAILED
> [junit-timeout] expected:<2> but was:<0>
> [junit-timeout] junit.framework.AssertionFailedError: expected:<2> but was:<0>
> [junit-timeout] at 
> org.apache.cassandra.utils.binlog.BinLogTest.testBinLogStartStop(BinLogTest.java:172)
> [junit-timeout]
> [junit-timeout]
> [junit-timeout] Test org.apache.cassandra.utils.binlog.BinLogTest FAILED
> {code}
> Reproducer
> {code}
> PASSED=0; time  { while ant testclasslist -Dtest.classlistprefix=unit 
> -Dtest.classlistfile=<(echo 
> org/apache/cassandra/utils/binlog/BinLogTest.java); do PASSED=$((PASSED+1)); 
> echo PASSED $PASSED; done }; echo FAILED after $PASSED runs.
> {code}
> In the last four attempts it has taken 31, 38, 27 and 10 rounds respectively 
> under JDK11 and took 51 under JDK8 (about 15 minutes).
> I have not tried running in a cpu-limited container or anything like that yet.
> Additionally, this went past in the logs a few times (under JDK11).  No idea 
> if it's just an artifact of weird test setup, or something more serious.
> {code}
> [junit-timeout] WARNING: Please consider reporting this to the maintainers of 
> net.openhft.chronicle.core.Jvm
> [junit-timeout] WARNING: Use --illegal-access=warn to enable warnings of 
> further illegal reflective access operations
> [junit-timeout] WARNING: All illegal access operations will be denied in a 
> future release
> [junit-timeout] Testsuite: org.apache.cassandra.utils.binlog.BinLogTest Tests 
> run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.839 sec
> [junit-timeout]
> [junit-timeout] java.lang.Throwable: 1e53135d-main creation ref-count=1
> [junit-timeout] at 
>

[jira] [Comment Edited] (CASSANDRASC-23) Set up structure for handling multiple Cassandra versions



[ 
https://issues.apache.org/jira/browse/CASSANDRASC-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106449#comment-17106449
 ] 

Jon Haddad edited comment on CASSANDRASC-23 at 5/13/20, 4:44 PM:
-

PR: [https://github.com/apache/cassandra-sidecar/pull/14]

This patch adds support for multiple Cassandra versions by decoupling the 
Cassandra logic from the VertX server.  

*The Highlights*:
 * The project is now broken into a few logical submodules. A common one sits 
at the base, providing driver wrapper and the framework for wrapping the driver
 * A Cassandra Interface, {{ICassandraAdapter}}, will provide the abstraction 
by which we interact with Cassandra going forward. I've only added a simple 
call - {{getStatus()}}, which serves only as a placeholder to establish working 
tests.
 * A wrapper for the ICassandra adapter, {{CassandraAdapterDelegate}}, which 
also implements the ICassandraAdapter interface, handles picking the right 
Adapter for a given connection. The rest services only need to be given an 
instance of the delegate, it will automatically pick the right Cassandra 
version and delegate all calls to the underlying version. If Cassandra is 
restarted (or upgraded), it will swap out the underlying implementation based 
on the version it detects when the session is available agin. This is 
completely hidden from the REST services.
 * A Test Template Framework, {{CassandraTestTemplate}} supported by docker via 
TestContainers, is provided for integration testing. It can be used by 
annotating a test with {{@CassandraIntegrationTest}}. One instance of the test 
will be created for each version of Cassandra we want to test against.  A 
{{CassandraTestContext}} is injected into each test providing access to the 
session, container, and the {{ICassandraAdapter}} for this version.
 * A bare bones 4.0 implementation is provided. Other implementations can be 
plugged in by overriding the Guice injector, so if a team is running a fork 
with custom functionality, it can be used.
 * Driver was upgraded to 3.9 for JDK 11 compatibility. Otherwise exceptions 
are thrown due to reflection usage in Netty.
 * The docs build was failing because it was running in a Docker container. 
I've updated the circle ci config to only generate docs in the docs build, 
otherwise it fails.
 * The REST service can (and does) mock the results of the Cassandra Delegate. 
This allows us to test every result coming back from Cassandra.

*Note*: I still need to update the developer documentation, it's a bit lacking 
in details.

*Follow up work*:
 * The test template references the 4.0 implementation directly. I'd like to 
set up a bit of service discovery or allow the configuration to be specified 
via a system property, I lean toward the latter (less magic), but I don't hold 
that view very strongly at all.
 * The circle ci configuration should be updated to cache the tarballs between 
runs so they don't need to be re-downloaded every time. I've added a 
{{sidecar.tarballs}} property that controls where the tarballs are cached, we 
just need to set this and cache the location.
 * We need to determine the structure for {{ICassandraAdapter}}. I lean towards 
composing together small interfaces (ICompaction, IRepair, etc) so we can keep 
it from becoming a giant megaclass. I believe that's best done outside this 
ticket.


was (Author: rustyrazorblade):
PR: https://github.com/apache/cassandra-sidecar/pull/14

This patch adds support for multiple Cassandra versions by decoupling the 
Cassandra logic from the VertX server.  

*The Highlights*:
 * The project is now broken into a few logical submodules. A common one sits 
at the base, providing driver wrapper and the framework for wrapping the driver
 * A Cassandra Interface, {{ICassandraAdapter}}, will provide the abstraction 
by which we interact with Cassandra going forward. I've only added a simple 
call - {{getStatus()}}, which serves only as a placeholder to establish working 
tests.
 * A wrapper for the ICassandra adapter, {{CassandraAdapterDelegate}}, which 
also implements the ICassandraAdapter interface, handles picking the right 
Adapter for a given connection. The rest services only need to be given an 
instance of the delegate, it will automatically pick the right Cassandra 
version and delegate all calls to the underlying version. If Cassandra is 
restarted (or upgraded), it will swap out the underlying implementation based 
on the version it detects when the session is available agin. This is 
completely hidden from the REST services.
 * A Test Template Framework, {{CassandraTestTemplate}} supported by docker via 
TestContainers, is provided for integration testing. It can be used by 
annotating a test with {{@CassandraIntegrationTest}}. One instance of the test 
will be created for each version of Cassandra we want to test against.
 * A bare bones 4.0 implementation is

[jira] [Commented] (CASSANDRA-15533) Don't allocate unneeded MergeIterator in OnDiskToken#iterator

2020-05-13 Thread Jordan West (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106458#comment-17106458
 ] 

Jordan West commented on CASSANDRA-15533:
-

Ack. Thanks [~blerer]

> Don't allocate unneeded MergeIterator in OnDiskToken#iterator 
> --
>
> Key: CASSANDRA-15533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15533
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.x
>
>
> When reviewing CASSANDRA-15392 it became apparent that the MergeIterator 
> allocated by OnDiskToken#iterator is rarely necessary and so we should avoid 
> allocating one when not needed and skip the MergeIterator pool when needed 
> because its unlikely to be sized correctly. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRASC-23) Set up structure for handling multiple Cassandra versions



 [ 
https://issues.apache.org/jira/browse/CASSANDRASC-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRASC-23:
--
Reviewers: Dinesh Joshi

> Set up structure for handling multiple Cassandra versions
> -
>
> Key: CASSANDRASC-23
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-23
> Project: Sidecar for Apache Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>Priority: Normal
>
> The first sidecar release will be for Cassandra 4.0, but one of the project 
> goals is to be able to handle multiple versions.  This will be especially 
> important in mixed version clusters, or even mixed version 1:N sidecar to C* 
> nodes (see CASSANDRASC-17).
> This JIRA is to lay the foundational work for supporting multiple Cassandra 
> versions. 
> Each Cassandra version (for example \{{cassandra40}}, \{{cassandra50}}, will 
> be in a separate Gradle subproject, implementing a common interface defined 
> in a \{{common}} subproject.  A common cassandra integration testing 
> framework will be able to test every version to ensure each version adheres 
> to the expected interface.
> Each module will define the minimum compatibility it supports, so if someone 
> (for example) wants to implement a Cassandra30 module for their own cluster, 
> they will have the ability to do so.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRASC-23) Set up structure for handling multiple Cassandra versions

[
https://issues.apache.org/jira/browse/CASSANDRASC-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106449#comment-17106449
]

Jon Haddad edited comment on CASSANDRASC-23 at 5/13/20, 4:38 PM:
-

PR: https://github.com/apache/cassandra-sidecar/pull/14

This patch adds support for multiple Cassandra versions by decoupling the
Cassandra logic from the VertX server.

*The Highlights*:
* The project is now broken into a few logical submodules. A common one sits
at the base, providing driver wrapper and the framework for wrapping the driver
* A Cassandra Interface, {{ICassandraAdapter}}, will provide the abstraction
by which we interact with Cassandra going forward. I've only added a simple
call - {{getStatus()}}, which serves only as a placeholder to establish working
tests.
* A wrapper for the ICassandra adapter, {{CassandraAdapterDelegate}}, which
also implements the ICassandraAdapter interface, handles picking the right
Adapter for a given connection. The rest services only need to be given an
instance of the delegate, it will automatically pick the right Cassandra
version and delegate all calls to the underlying version. If Cassandra is
restarted (or upgraded), it will swap out the underlying implementation based
on the version it detects when the session is available agin. This is
completely hidden from the REST services.
* A Test Template Framework, {{CassandraTestTemplate}} supported by docker via
TestContainers, is provided for integration testing. It can be used by
annotating a test with {{@CassandraIntegrationTest}}. One instance of the test
will be created for each version of Cassandra we want to test against.
* A bare bones 4.0 implementation is provided. Other implementations can be
plugged in by overriding the Guice injector, so if a team is running a fork
with custom functionality, it can be used.
* Driver was upgraded to 3.9 for JDK 11 compatibility. Otherwise exceptions
are thrown due to reflection usage in Netty.
* The docs build was failing because it was running in a Docker container.
I've updated the circle ci config to only generate docs in the docs build,
otherwise it fails.
* The REST service can (and does) mock the results of the Cassandra Delegate.
This allows us to test every result coming back from Cassandra.

*Note*: I still need to update the developer documentation, it's a bit lacking
in details.

*Follow up work*:
* The test template references the 4.0 implementation directly. I'd like to
set up a bit of service discovery or allow the configuration to be specified
via a system property, I lean toward the latter (less magic), but I don't hold
that view very strongly at all.
* The circle ci configuration should be updated to cache the tarballs between
runs so they don't need to be re-downloaded every time. I've added a
{{sidecar.tarballs}} property that controls where the tarballs are cached, we
just need to set this and cache the location.
* We need to determine the structure for {{ICassandraAdapter}}. I lean towards
composing together small interfaces (ICompaction, IRepair, etc) so we can keep
it from becoming a giant megaclass. I believe that's best done outside this
ticket.

was (Author: rustyrazorblade):
This patch adds support for multiple Cassandra versions by decoupling the
Cassandra logic from the VertX server.

*The Highlights*:

* The project is now broken into a few logical submodules. A common one sits
at the base, providing driver wrapper and the framework for wrapping the driver
* A Cassandra Interface, {{ICassandraAdapter}}, will provide the abstraction by
which we interact with Cassandra going forward. I've only added a simple call
- {{getStatus()}}, which serves only as a placeholder to establish working
tests.
* A wrapper for the ICassandra adapter, {{CassandraAdapterDelegate}}, which
also implements the ICassandraAdapter interface, handles picking the right
Adapter for a given connection. The rest services only need to be given an
instance of the delegate, it will automatically pick the right Cassandra
version and delegate all calls to the underlying version. If Cassandra is
restarted (or upgraded), it will swap out the underlying implementation based
on the version it detects when the session is available agin. This is
completely hidden from the REST services.
* A Test Template Framework, {{CassandraTestTemplate}} supported by docker via
TestContainers, is provided for integration testing. It can be used by
annotating a test with {{@CassandraIntegrationTest}}. One instance of the test
will be created for each version of Cassandra we want to test against.
* A bare bones 4.0 implementation is provided. Other implementations can be
plugged in by overriding the Guice injector, so if a team is running a fork
with custom functionality, it can be used.
* Driver was upgraded to 3.9 for JDK 11

[jira] [Updated] (CASSANDRASC-23) Set up structure for handling multiple Cassandra versions



 [ 
https://issues.apache.org/jira/browse/CASSANDRASC-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRASC-23:
--
Authors: Jon Haddad
Test and Documentation Plan: 
[CircleCI|https://app.circleci.com/pipelines/github/rustyrazorblade/cassandra-sidecar?branch=jon%2F23%2Fmultiple-cassandra-versions]


 Status: Patch Available  (was: Open)

This patch adds support for multiple Cassandra versions by decoupling the 
Cassandra logic from the VertX server.  

*The Highlights*:

* The project is now broken into a few logical submodules.  A common one sits 
at the base, providing driver wrapper and the framework for wrapping the driver
* A Cassandra Interface, {{ICassandraAdapter}}, will provide the abstraction by 
which we interact with Cassandra going forward.  I've only added a simple call 
- {{getStatus()}}, which serves only as a placeholder to establish working 
tests.
* A wrapper for the ICassandra adapter, {{CassandraAdapterDelegate}}, which 
also implements the ICassandraAdapter interface, handles picking the right 
Adapter for a given connection.  The rest services only need to be given an 
instance of the delegate, it will automatically pick the right Cassandra 
version and delegate all calls to the underlying version.  If Cassandra is 
restarted (or upgraded), it will swap out the underlying implementation based 
on the version it detects when the session is available agin.  This is 
completely hidden from the REST services.
* A Test Template Framework, {{CassandraTestTemplate}} supported by docker via 
TestContainers, is provided for integration testing.  It can be used by 
annotating a test with {{@CassandraIntegrationTest}}.  One instance of the test 
will be created for each version of Cassandra we want to test against.  
* A bare bones 4.0 implementation is provided.  Other implementations can be 
plugged in by overriding the Guice injector, so if a team is running a fork 
with custom functionality, it can be used.
* Driver was upgraded to 3.9 for JDK 11 compatibility.  Otherwise exceptions 
are thrown due to reflection usage in Netty.
* The docs build was failing because it was running in a Docker container.  
I've updated the circle ci config to only generate docs in the docs build, 
otherwise it fails.
* The REST service can (and does) mock the results of the Cassandra Delegate.  
This allows us to test every result coming back from Cassandra.

*Note*: I still need to update the developer documentation, it's a bit lacking 
in details.

*Follow up work*:

* The test template references the 4.0 implementation directly.  I'd like to 
set up a bit of service discovery or allow the configuration to be specified 
via a system property, I lean toward the latter (less magic), but I don't hold 
that view very strongly at all.
* The circle ci configuration should be updated to cache the tarballs between 
runs so they don't need to be re-downloaded every time.  I've added a 
{{sidecar.tarballs}} property that controls where the tarballs are cached, we 
just need to set this and cache the location.
* We need to determine the structure for {{ICassandraAdapter}}.  I lean towards 
composing together small interfaces (ICompaction, IRepair, etc) so we can keep 
it from becoming a giant megaclass.  I believe that's best done outside this 
ticket.


> Set up structure for handling multiple Cassandra versions
> -
>
> Key: CASSANDRASC-23
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-23
> Project: Sidecar for Apache Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>Priority: Normal
>
> The first sidecar release will be for Cassandra 4.0, but one of the project 
> goals is to be able to handle multiple versions.  This will be especially 
> important in mixed version clusters, or even mixed version 1:N sidecar to C* 
> nodes (see CASSANDRASC-17).
> This JIRA is to lay the foundational work for supporting multiple Cassandra 
> versions. 
> Each Cassandra version (for example \{{cassandra40}}, \{{cassandra50}}, will 
> be in a separate Gradle subproject, implementing a common interface defined 
> in a \{{common}} subproject.  A common cassandra integration testing 
> framework will be able to test every version to ensure each version adheres 
> to the expected interface.
> Each module will define the minimum compatibility it supports, so if someone 
> (for example) wants to implement a Cassandra30 module for their own cluster, 
> they will have the ability to do so.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For

[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15791:
---
  Since Version: 4.0-alpha
Source Control Link: 
https://github.com/apache/cassandra/commit/a209e78dae8674ef131bab1b2dc06f09ae15d21d
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed into trunk at a209e78dae8674ef131bab1b2dc06f09ae15d21d 

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Fix unguarded casts to NetworkTopologyStrategy

2020-05-13 Thread blerer

This is an automated email from the ASF dual-hosted git repository.

blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new a209e78  Fix unguarded casts to NetworkTopologyStrategy
a209e78 is described below

commit a209e78dae8674ef131bab1b2dc06f09ae15d21d
Author: Bereng 
AuthorDate: Thu May 7 16:53:02 2020 +0200

Fix unguarded casts to NetworkTopologyStrategy

patch by  Berenguer Blasi; reviewed by Benjamin Lerer for CASSANDRA-15791
---
 .../org/apache/cassandra/db/ConsistencyLevel.java| 20 +++-
 .../service/DatacenterSyncWriteResponseHandler.java  | 16 +++-
 2 files changed, 26 insertions(+), 10 deletions(-)

diff --git a/src/java/org/apache/cassandra/db/ConsistencyLevel.java 
b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
index 91e83a7..e3da5b3 100644
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@ -100,11 +100,21 @@ public enum ConsistencyLevel
 
 public static ObjectIntHashMap eachQuorumForRead(Keyspace keyspace)
 {
-NetworkTopologyStrategy strategy = (NetworkTopologyStrategy) 
keyspace.getReplicationStrategy();
-ObjectIntHashMap perDc = new 
ObjectIntHashMap<>(((strategy.getDatacenters().size() + 1) * 4) / 3);
-for (String dc : strategy.getDatacenters())
-perDc.put(dc, ConsistencyLevel.localQuorumFor(keyspace, dc));
-return perDc;
+AbstractReplicationStrategy strategy = 
keyspace.getReplicationStrategy();
+if (strategy instanceof NetworkTopologyStrategy)
+{
+NetworkTopologyStrategy npStrategy = (NetworkTopologyStrategy) 
strategy;
+ObjectIntHashMap perDc = new 
ObjectIntHashMap<>(((npStrategy.getDatacenters().size() + 1) * 4) / 3);
+for (String dc : npStrategy.getDatacenters())
+perDc.put(dc, ConsistencyLevel.localQuorumFor(keyspace, dc));
+return perDc;
+}
+else
+{
+ObjectIntHashMap perDc = new ObjectIntHashMap<>(1);
+perDc.put(DatabaseDescriptor.getLocalDataCenter(), 
quorumFor(keyspace));
+return perDc;
+}
 }
 
 public static ObjectIntHashMap eachQuorumForWrite(Keyspace 
keyspace, Endpoints pendingWithDown)
diff --git 
a/src/java/org/apache/cassandra/service/DatacenterSyncWriteResponseHandler.java 
b/src/java/org/apache/cassandra/service/DatacenterSyncWriteResponseHandler.java
index 1f536c7..389dcd5 100644
--- 
a/src/java/org/apache/cassandra/service/DatacenterSyncWriteResponseHandler.java
+++ 
b/src/java/org/apache/cassandra/service/DatacenterSyncWriteResponseHandler.java
@@ -49,12 +49,18 @@ public class DatacenterSyncWriteResponseHandler extends 
AbstractWriteResponse
 super(replicaPlan, callback, writeType, queryStartNanoTime);
 assert replicaPlan.consistencyLevel() == ConsistencyLevel.EACH_QUORUM;
 
-NetworkTopologyStrategy strategy = (NetworkTopologyStrategy) 
replicaPlan.keyspace().getReplicationStrategy();
-
-for (String dc : strategy.getDatacenters())
+if (replicaPlan.keyspace().getReplicationStrategy() instanceof 
NetworkTopologyStrategy)
+{
+NetworkTopologyStrategy strategy = (NetworkTopologyStrategy) 
replicaPlan.keyspace().getReplicationStrategy();
+for (String dc : strategy.getDatacenters())
+{
+int rf = strategy.getReplicationFactor(dc).allReplicas;
+responses.put(dc, new AtomicInteger((rf / 2) + 1));
+}
+}
+else
 {
-int rf = strategy.getReplicationFactor(dc).allReplicas;
-responses.put(dc, new AtomicInteger((rf / 2) + 1));
+responses.put(DatabaseDescriptor.getLocalDataCenter(), new 
AtomicInteger(ConsistencyLevel.quorumFor(replicaPlan.keyspace(;
 }
 
 // During bootstrap, we have to include the pending endpoints or we 
may fail the consistency level


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15799) CorruptSSTableException when compacting a 3.0 format sstable that was originally created as an outcome of 2.1 sstable upgrade

2020-05-13 Thread Sumanth Pasupuleti (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106424#comment-17106424
 ] 

Sumanth Pasupuleti commented on CASSANDRA-15799:


[~dcapwell] So I applied your POC along with CASSANDRA-15373 on top of 3.0.19, 
and below is the behavior now

{code:java}
cqlsh> select *from turbo."VersionedCollectionContents";
ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
message="Operation failed - received 0 responses and 1 failures" 
info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
'consistency': 'ONE'}
{code}


{code:java}
ERROR [SharedPool-Worker-2] 2020-05-13 16:00:19,090 
AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
Thread[SharedPool-Worker-2,5,main]
java.lang.RuntimeException: org.apache.cassandra.serializers.MarshalException: 
Expected 8 or 0 byte long (14394)
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_231]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
 [nf-cassandra-3.0.19.8.jar:3.0.19.8]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_231]
Caused by: org.apache.cassandra.serializers.MarshalException: Expected 8 or 0 
byte long (14394)
at 
org.apache.cassandra.serializers.LongSerializer.validate(LongSerializer.java:42)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.marshal.AbstractType.validate(AbstractType.java:159) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.marshal.AbstractType.validateIfFixedSize(AbstractType.java:390)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.LegacyLayout.decodeClustering(LegacyLayout.java:456) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.LegacyLayout.decodeCellName(LegacyLayout.java:162) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.LegacyLayout.readLegacyCellBody(LegacyLayout.java:1227) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.LegacyLayout.readLegacyAtom(LegacyLayout.java:1187) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.readAtom(UnfilteredDeserializer.java:294)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator$AtomIterator.hasNext(UnfilteredDeserializer.java:601)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:500)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:336)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:140)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:172)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:336)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:220)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:33)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
 ~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
~[nf-cassandra-3.0.19.8.jar:3.0.19.8]
at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)

[jira] [Created] (CASSANDRA-15811) Improve DROP COMPACT STORAGE

Alex Petrov created CASSANDRA-15811:
---

 Summary: Improve DROP COMPACT STORAGE
 Key: CASSANDRA-15811
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15811
 Project: Cassandra
  Issue Type: Bug
Reporter: Alex Petrov


[TBD]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15810) Default StringTableSize parameter causes GC slowdown

2020-05-13 Thread Tom van der Woerdt (Jira)

Tom van der Woerdt created CASSANDRA-15810:
--

 Summary: Default StringTableSize parameter causes GC slowdown
 Key: CASSANDRA-15810
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15810
 Project: Cassandra
  Issue Type: Improvement
Reporter: Tom van der Woerdt


While looking at tail latency on a Cassandra cluster, it came up that the 
default StringTableSize in Cassandra is set to a million:
{code:java}
# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
-XX:StringTableSize=103{code}
This was done for CASSANDRA-6410 by [~jbellis] in '13, to optimize heap usage 
on a test case, running with 500 nodes and num_tokens=512.

Until Java 13, this string table is implemented as native code, and has to be 
traversed entirely during the GC initial marking phase, which is a STW event.

Some testing on my end shows that the pause time of a GC cycle can be reduced 
by approximately 10 milliseconds if we lower the string table size back to the 
Java 8 default of 60013 entries.

Thus, I would recommend this patch (3.11 branch, similar patch for 4.0):
{code:java}
diff --git a/conf/jvm.options b/conf/jvm.options
index 01bb1685b3..c184d18c5d 100644
--- a/conf/jvm.options
+++ b/conf/jvm.options
@@ -107,9 +107,6 @@
 # Per-thread stack size.
 -Xss256k

-# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
--XX:StringTableSize=103
-
 # Make sure all memory is faulted and zeroed on startup.
 # This helps prevent soft faults in containers and makes
 # transparent hugepage allocation more effective.
 {code}
It does need some testing on more extreme clusters than I have access to, but I 
ran some Cassandra nodes with {{-XX:+PrintStringTableStatistics}} which 
suggested that the Java default will suffice here.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15727:
--
Authors: Andy Tolbert, Jon Meredith  (was: Jon Meredith)

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15727:
--
Status: Ready to Commit  (was: Review In Progress)

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15727:
--
Fix Version/s: (was: 4.0-beta)
   4.0-alpha

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15727:
--
  Since Version: 4.0-alpha
Source Control Link: 
[678ca3fc29c38b64a110dcf40693aa7840b0585c|https://github.com/apache/cassandra/commit/678ca3fc29c38b64a110dcf40693aa7840b0585c]
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Cheers, committed as 
[678ca3fc29c38b64a110dcf40693aa7840b0585c|https://github.com/apache/cassandra/commit/678ca3fc29c38b64a110dcf40693aa7840b0585c]
 to trunk.

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Update port when reconnecting to pre-4.0 SSL storage

2020-05-13 Thread aleksey

This is an automated email from the ASF dual-hosted git repository.

aleksey pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 678ca3f  Update port when reconnecting to pre-4.0 SSL storage
678ca3f is described below

commit 678ca3fc29c38b64a110dcf40693aa7840b0585c
Author: Jon Meredith 
AuthorDate: Tue Apr 7 18:58:59 2020 -0600

Update port when reconnecting to pre-4.0 SSL storage

On a failed outbound connection to a node with pending data, recheck
the messaging version before reattempting the connection.

Prior to this change, if the endpoint version was incorrectly set
to 4.0 when the node was running 3.0 with an SSL storage port
the connection would continuously try to reconnect on the wrong port.

The patch also improves some of the log messages to include the
actual port being connected to as well as the canonical endpoint for
the node.

Patch by Jon Meredith & Andy Tolbert; reviewed by Aleksey Yeschenko for
CASSANDRA-15727

Co-authored-by: Jon Meredith 
Co-authored-by: Andy Tolbert 
---
 CHANGES.txt|  1 +
 .../cassandra/net/InboundConnectionInitiator.java  |  4 +-
 .../org/apache/cassandra/net/InboundSockets.java   | 19 -
 .../apache/cassandra/net/OutboundConnection.java   | 20 -
 .../cassandra/net/OutboundConnectionInitiator.java | 10 +--
 .../cassandra/net/OutboundConnectionSettings.java  |  7 ++
 .../org/apache/cassandra/net/ConnectionTest.java   | 92 ++
 7 files changed, 142 insertions(+), 11 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index e6acf40..98b3f68 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-alpha5
+ * Update port when reconnecting to pre-4.0 SSL storage (CASSANDRA-15727)
  * Only calculate dynamicBadnessThreshold once per loop in 
DynamicEndpointSnitch (CASSANDRA-15798)
  * Cleanup redundant nodetool commands added in 4.0 (CASSANDRA-15256)
  * Update to Python driver 3.23 for cqlsh (CASSANDRA-15793)
diff --git a/src/java/org/apache/cassandra/net/InboundConnectionInitiator.java 
b/src/java/org/apache/cassandra/net/InboundConnectionInitiator.java
index c390ba4..3c1498b 100644
--- a/src/java/org/apache/cassandra/net/InboundConnectionInitiator.java
+++ b/src/java/org/apache/cassandra/net/InboundConnectionInitiator.java
@@ -239,8 +239,8 @@ public class InboundConnectionInitiator
 if (sslHandler != null)
 {
 SSLSession session = sslHandler.engine().getSession();
-logger.info("connection from peer {}, protocol = {}, cipher 
suite = {}",
-ctx.channel().remoteAddress(), 
session.getProtocol(), session.getCipherSuite());
+logger.info("connection from peer {} to {}, protocol = {}, 
cipher suite = {}",
+ctx.channel().remoteAddress(), 
ctx.channel().localAddress(), session.getProtocol(), session.getCipherSuite());
 }
 }
 
diff --git a/src/java/org/apache/cassandra/net/InboundSockets.java 
b/src/java/org/apache/cassandra/net/InboundSockets.java
index 8f74eaa..eb9ef8e 100644
--- a/src/java/org/apache/cassandra/net/InboundSockets.java
+++ b/src/java/org/apache/cassandra/net/InboundSockets.java
@@ -187,10 +187,23 @@ class InboundSockets
 
 private static void addBindings(InboundConnectionSettings template, 
ImmutableList.Builder out)
 {
-InboundConnectionSettings settings = template.withDefaults();
-out.add(new InboundSocket(settings));
+InboundConnectionSettings   settings = template.withDefaults();
+InboundConnectionSettings legacySettings = 
template.withLegacyDefaults();
+
 if (settings.encryption.enable_legacy_ssl_storage_port && 
settings.encryption.enabled)
-out.add(new InboundSocket(template.withLegacyDefaults()));
+{
+out.add(new InboundSocket(legacySettings));
+
+/*
+ * If the legacy ssl storage port and storage port match, only 
bind to the
+ * legacy ssl port. This makes it possible to configure a 4.0 node 
like a 3.0
+ * node with only the ssl_storage_port if required.
+ */
+if (settings.bindAddress.equals(legacySettings.bindAddress))
+return;
+}
+
+out.add(new InboundSocket(settings));
 }
 
 public Future open(Consumer pipelineInjector)
diff --git a/src/java/org/apache/cassandra/net/OutboundConnection.java 
b/src/java/org/apache/cassandra/net/OutboundConnection.java
index b84ebc3..315d086 100644
--- a/src/java/org/apache/cassandra/net/OutboundConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundConnection.java
@@ -1084,7 +1084,25 @@ public class OutboundConnection
 if (hasPending())
 {
 Promise>

[jira] [Commented] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106397#comment-17106397
 ] 

Michael Semb Wever commented on CASSANDRA-15776:


New ASF CI run is 
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/121/pipeline]

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106397#comment-17106397
 ] 

Michael Semb Wever edited comment on CASSANDRA-15776 at 5/13/20, 3:26 PM:
--

Has a few small nits/questions on naming, see PR.

New ASF CI run is 
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/121/pipeline]


was (Author: michaelsembwever):
New ASF CI run is 
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/121/pipeline]

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106394#comment-17106394
 ] 

Benjamin Lerer commented on CASSANDRA-15791:


The patch looks good to me.

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15791:
---
Status: Ready to Commit  (was: Review In Progress)

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15791:
---
Reviewers: Benjamin Lerer, Benjamin Lerer  (was: Benjamin Lerer)
   Benjamin Lerer, Benjamin Lerer  (was: Benjamin Lerer)
   Status: Review In Progress  (was: Patch Available)

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect

2020-05-13 Thread Jon Meredith (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106357#comment-17106357
 ] 

Jon Meredith commented on CASSANDRA-15727:
--

Thanks for the review.  The fix looks necessary to me, and the nits are all 
good. Please merge when you're happy with it.

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106331#comment-17106331
 ] 

Aleksey Yeschenko commented on CASSANDRA-15727:
---

The patch mostly looks good to me, but I think there is an issue with the logic 
in `InboundSockets#addBindings()` - I *think* it's possible for the ports to 
match in config but for encryption or legacy ssl storage port to not be 
enabled, thus never adding any bindings at all.

Also some formatting nits and unnecessary `this` that we usually avoid.

Pushed [here|https://github.com/iamaleksey/cassandra/commits/15727-4.0] - 
please have a look, as I'm not certain about issue 1.

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106331#comment-17106331
 ] 

Aleksey Yeschenko edited comment on CASSANDRA-15727 at 5/13/20, 2:08 PM:
-

The patch mostly looks good to me, but I think there is an issue with the logic 
in {{InboundSockets#addBindings()}} - I *think* it's possible for the ports to 
match in config but for encryption or legacy ssl storage port to not be 
enabled, thus never adding any bindings at all.

Also some formatting nits and unnecessary `this` that we usually avoid.

Pushed [here|https://github.com/iamaleksey/cassandra/commits/15727-4.0] - 
please have a look, as I'm not certain about issue 1.


was (Author: iamaleksey):
The patch mostly looks good to me, but I think there is an issue with the logic 
in `InboundSockets#addBindings()` - I *think* it's possible for the ports to 
match in config but for encryption or legacy ssl storage port to not be 
enabled, thus never adding any bindings at all.

Also some formatting nits and unnecessary `this` that we usually avoid.

Pushed [here|https://github.com/iamaleksey/cassandra/commits/15727-4.0] - 
please have a look, as I'm not certain about issue 1.

> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if 
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0 
> to 4.0.  The 3.0 cluster was configured to only listen on the ssl storage 
> port. When the upgraded 4.0 node started it received a gossip messsage that 
> triggered a shadow round before it had correctly set the messaging versions 
> for the other endpoints.
> Sending the message created the connection, but because the endpoint 
> defaulted to {{VERSION_40}} the initial connect attempt was to the regular 
> {{storage_port}}.  The 3.0 node was only listening on the 
> {{ssl_storage_port}}, so the connection was refused and the 
> {{OutboundConnection.onFailure}} handler was called.  As the shadow
> gossip round had queued up a message, the {{hasPending}} branch was followed 
> and the connection was rescheduled, however the port is never recalculated as 
> the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound 
> connection and gossip updating the messaging version for the endpoint which 
> could have been used to make a valid connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15582) 4.0 quality testing: metrics

2020-05-13 Thread Stephen Mallette (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106324#comment-17106324
 ] 

Stephen Mallette commented on CASSANDRA-15582:
--

> How do we ensure that all code paths that generate the metrics are exercised?

>> I think this was a good point as well. I've yet to come across a metric that 
>> wasn't initialized as start of cassandra

After a bit of digging it seems that there are a number of metrics that don't 
simply initialize as a result of cassandra starting up successfully. 

* Streaming Metrics
* HintedHandoff Metrics
* HintsService Metrics
* Batch Metrics

It's also worth noting that certain metric scopes don't seem to initialize on 
startup, specifically: ThreadPools. As a result there a number of missing 
metrics to evaluate.

It seems that to cover everything we'd need to fire some sort of "metric 
initialization" script prior to gathering metrics for analysis/testing. I guess 
I will start researching how best to do that. Of course, my knowledge of what's 
missing comes from the documentation being compared to the results of my jmx 
scripts so if there is a metric not in the docs AND also not initialized at 
cassandra start, I don't know how to uncover that except by way of digging 
through code.



> 4.0 quality testing: metrics
> 
>
> Key: CASSANDRA-15582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15582
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Romain Hardouin
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Screen Shot 2020-04-07 at 5.47.17 PM.png
>
>
> In past releases we've unknowingly broken metrics integrations and introduced 
> performance regressions in metrics collection and reporting. We strive in 4.0 
> to not do that. Metrics should work well!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15791:
---
Reviewers: Benjamin Lerer  (was: Andres de la Peña)

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15805:
--
Reviewers: Aleksey Yeschenko, Marcus Eriksson, Aleksey Yeschenko  (was: 
Aleksey Yeschenko, Marcus Eriksson)
   Aleksey Yeschenko, Marcus Eriksson, Aleksey Yeschenko  (was: 
Aleksey Yeschenko, Marcus Eriksson)
   Status: Review In Progress  (was: Patch Available)

> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones 
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The legacy reading code ({{LegacyLayout}} and 
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly 
> the case where a range tombstone covering multiple rows interacts with a 
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
>   k int,
>   c1 text,
>   c2 text,
>   a text,
>   b set,
>   c text,
>   PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'}, 
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same 
> table or compacted together), then this will result in the inserted row being 
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this 
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said, 
> the additional code added to CASSANDRA-15789 to force merging duplicated rows 
> if they are produced _will_ end up fixing this as a consequence (assuming 
> there is no variation of this problem that leads to other visible issues than 
> duplicated rows). That said, I "think" we'd still rather fix the source of 
> the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15805:
--
Test and Documentation Plan: in-jvm dtests included
 Status: Patch Available  (was: Open)

> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones 
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The legacy reading code ({{LegacyLayout}} and 
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly 
> the case where a range tombstone covering multiple rows interacts with a 
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
>   k int,
>   c1 text,
>   c2 text,
>   a text,
>   b set,
>   c text,
>   PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'}, 
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same 
> table or compacted together), then this will result in the inserted row being 
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this 
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said, 
> the additional code added to CASSANDRA-15789 to force merging duplicated rows 
> if they are produced _will_ end up fixing this as a consequence (assuming 
> there is no variation of this problem that leads to other visible issues than 
> duplicated rows). That said, I "think" we'd still rather fix the source of 
> the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-15778:
---

Assignee: a  (was: David Capwell)

> CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
> -
>
> Key: CASSANDRA-15778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15778
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: a
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is consistently 
> reproduce-able.
> {code:java}
> ERROR [SharedPool-Worker-1] 2020-05-01 14:57:57,661 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,main]ERROR [SharedPool-Worker-1] 2020-05-01 
> 14:57:57,661 AbstractLocalAwareExecutorService.java:169 - Uncaught exception 
> on thread 
> Thread[SharedPool-Worker-1,5,main]org.apache.cassandra.io.sstable.CorruptSSTableException:
>  Corrupted: 
> /mnt/data/cassandra/data//  at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:349)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:220)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:33)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:294)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:341) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:47)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_231] at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
>

[jira] [Assigned] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-15778:
---

Assignee: Alex Petrov  (was: a)

> CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
> -
>
> Key: CASSANDRA-15778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15778
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is consistently 
> reproduce-able.
> {code:java}
> ERROR [SharedPool-Worker-1] 2020-05-01 14:57:57,661 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,main]ERROR [SharedPool-Worker-1] 2020-05-01 
> 14:57:57,661 AbstractLocalAwareExecutorService.java:169 - Uncaught exception 
> on thread 
> Thread[SharedPool-Worker-1,5,main]org.apache.cassandra.io.sstable.CorruptSSTableException:
>  Corrupted: 
> /mnt/data/cassandra/data//  at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:349)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:220)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:33)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:294)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:341) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:47)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_231] at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
>  ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at 
>

[jira] [Updated] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15784:
---
Component/s: (was: CI)
 Test/dtest

> test_refresh_size_estimates_clears_invalid_entries - 
> nodetool_test.TestNodetool
> ---
>
> Key: CASSANDRA-15784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15784
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> Dtest failure



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones

2020-05-13 Thread Sylvain Lebresne (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106271#comment-17106271
 ] 

Sylvain Lebresne commented on CASSANDRA-15805:
--

There is probably a few variations for how to fix this, but what feels the more 
intuitive to me is to:
# slightly modify the ctor of {{LegacyRangeTombstone}} so the {{atom5}} of my 
previous comment use an inclusive start. Mostly because that make the rest of 
the logic a bit simpler imo (we can still assume that when we get a atom whose 
cluster strictly sort after the currently grouper row, we're done with that 
row).
# modify {{UnfilteredDeserializer.OldFormatDeserializer}} so that when, while 
grouping a row, it encounters a RT that covers it, is "splits" that into a row 
tombstone covering the row, and push back the handling of the rest of the 
tombstone to when the row is truly finished.

I've pushed a patch doing so for 3.0 below (thanks to [~markus] for triggering 
CI on this):
||branch||unit tests||dtests||jvm dtests||jvm upgrade dtest||
| https://github.com/pcmanus/cassandra/commits/C-15805-3.0 | 
[utests|https://circleci.com/gh/krummas/cassandra/3289] | 
[vnodes|https://circleci.com/gh/krummas/cassandra/3292] 
[no-vnodes|https://circleci.com/gh/krummas/cassandra/3293] | [jvm 
dtests|https://circleci.com/gh/krummas/cassandra/3290] | [upgrade 
dtests|https://circleci.com/gh/krummas/cassandra/3294] |

I'll note that the branch contains another small fix, that is a lot less 
important. Namely, [the return at the beginning of 
{{CellGrouper#addCollectionTombstone}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1516]
 should be {{true}}, not {{false}} as it currently is. The test in question 
checks if a collection tombstone happens to not be selected by the query we're 
decoding data for. If it isn't included, we can ignore the tombstone, so we 
can/should return, but not with {{false}} as that imply the row is finished, 
which it probably isn't.

Now the reason I say that last problem is less important is that in practice, 
only thrift queries should run into this (since CQL queries queries all column 
effectively) so even if we duplicate the row here, it won't matter when the 
result is converted back to thrift (besides, having a collection tombstone 
implies that this is a thrift query on a CQL table, which is dodgy in the first 
place). Anyway, the code is still obviously wrong and the fix trivial, so 
included it it nonetheless (in a separate commit).


> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones 
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The legacy reading code ({{LegacyLayout}} and 
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly 
> the case where a range tombstone covering multiple rows interacts with a 
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
>   k int,
>   c1 text,
>   c2 text,
>   a text,
>   b set,
>   c text,
>   PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'}, 
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same 
> table or compacted together), then this will result in the inserted row being 
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this 
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said, 
> the additional code added to CASSANDRA-15789 to force merging duplicated rows 
> if they are produced _will_ end up fixing this as a consequence (assuming 
> there is no variation of this problem that leads to other visible issues than 
> duplicated rows). That said, I "think" we'd still rather fix the source of 
> the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail:

[jira] [Comment Edited] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones

2020-05-13 Thread Sylvain Lebresne (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106271#comment-17106271
 ] 

Sylvain Lebresne edited comment on CASSANDRA-15805 at 5/13/20, 12:51 PM:
-

There is probably a few variations for how to fix this, but what feels the more 
intuitive to me is to:
 # slightly modify the ctor of {{LegacyRangeTombstone}} so the {{atom5}} of my 
previous comment use an inclusive start. Mostly because that make the rest of 
the logic a bit simpler imo (we can still assume that when we get a atom whose 
cluster strictly sort after the currently grouper row, we're done with that 
row).
 # modify {{UnfilteredDeserializer.OldFormatDeserializer}} so that when, while 
grouping a row, it encounters a RT that covers it, is "splits" that into a row 
tombstone covering the row, and push back the handling of the rest of the 
tombstone to when the row is truly finished.

I've pushed a patch doing so for 3.0 below (thanks to [~marcuse] for triggering 
CI on this):
||branch||unit tests||dtests||jvm dtests||jvm upgrade dtest||
|[https://github.com/pcmanus/cassandra/commits/C-15805-3.0]|[utests|https://circleci.com/gh/krummas/cassandra/3289]|[vnodes|https://circleci.com/gh/krummas/cassandra/3292]
 [no-vnodes|https://circleci.com/gh/krummas/cassandra/3293]|[jvm 
dtests|https://circleci.com/gh/krummas/cassandra/3290]|[upgrade 
dtests|https://circleci.com/gh/krummas/cassandra/3294]|

I'll note that the branch contains another small fix, that is a lot less 
important. Namely, [the return at the beginning of 
{{CellGrouper#addCollectionTombstone}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1516]
 should be {{true}}, not {{false}} as it currently is. The test in question 
checks if a collection tombstone happens to not be selected by the query we're 
decoding data for. If it isn't included, we can ignore the tombstone, so we 
can/should return, but not with {{false}} as that imply the row is finished, 
which it probably isn't.

Now the reason I say that last problem is less important is that in practice, 
only thrift queries should run into this (since CQL queries queries all column 
effectively) so even if we duplicate the row here, it won't matter when the 
result is converted back to thrift (besides, having a collection tombstone 
implies that this is a thrift query on a CQL table, which is dodgy in the first 
place). Anyway, the code is still obviously wrong and the fix trivial, so 
included it it nonetheless (in a separate commit).


was (Author: slebresne):
There is probably a few variations for how to fix this, but what feels the more 
intuitive to me is to:
# slightly modify the ctor of {{LegacyRangeTombstone}} so the {{atom5}} of my 
previous comment use an inclusive start. Mostly because that make the rest of 
the logic a bit simpler imo (we can still assume that when we get a atom whose 
cluster strictly sort after the currently grouper row, we're done with that 
row).
# modify {{UnfilteredDeserializer.OldFormatDeserializer}} so that when, while 
grouping a row, it encounters a RT that covers it, is "splits" that into a row 
tombstone covering the row, and push back the handling of the rest of the 
tombstone to when the row is truly finished.

I've pushed a patch doing so for 3.0 below (thanks to [~markus] for triggering 
CI on this):
||branch||unit tests||dtests||jvm dtests||jvm upgrade dtest||
| https://github.com/pcmanus/cassandra/commits/C-15805-3.0 | 
[utests|https://circleci.com/gh/krummas/cassandra/3289] | 
[vnodes|https://circleci.com/gh/krummas/cassandra/3292] 
[no-vnodes|https://circleci.com/gh/krummas/cassandra/3293] | [jvm 
dtests|https://circleci.com/gh/krummas/cassandra/3290] | [upgrade 
dtests|https://circleci.com/gh/krummas/cassandra/3294] |

I'll note that the branch contains another small fix, that is a lot less 
important. Namely, [the return at the beginning of 
{{CellGrouper#addCollectionTombstone}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1516]
 should be {{true}}, not {{false}} as it currently is. The test in question 
checks if a collection tombstone happens to not be selected by the query we're 
decoding data for. If it isn't included, we can ignore the tombstone, so we 
can/should return, but not with {{false}} as that imply the row is finished, 
which it probably isn't.

Now the reason I say that last problem is less important is that in practice, 
only thrift queries should run into this (since CQL queries queries all column 
effectively) so even if we duplicate the row here, it won't matter when the 
result is converted back to thrift (besides, having a collection tombstone 
implies that this is a thrift query on a CQL table, which is dodgy in the first 
place). Anyway, the code is still obviously wrong and the fix

[jira] [Updated] (CASSANDRA-15211) Remove BaseIterator.stopChild

2020-05-13 Thread Benedict Elliott Smith (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict Elliott Smith updated CASSANDRA-15211:
---
Fix Version/s: (was: 4.0-beta)
   (was: 4.0)
   4.x

> Remove BaseIterator.stopChild
> -
>
> Key: CASSANDRA-15211
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15211
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL/Interpreter
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> This concept was introduced in CASSANDRA-13482, and while it logically 
> resolves the bug, the “more correct” solution is to attach the functions of 
> the extending iterator to the iterator they are being absorbed into.  
> CASSANDRA-13482 introduced a slight error in its evaluation of exit 
> conditions that cropped up in CASSANDRA-14812, that could be avoided with 
> this formulation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106257#comment-17106257
 ] 

Aleksey Yeschenko commented on CASSANDRA-15299:
---

bq. Firstly, we think that not all users will be interested in enabling 
checksumming. Making it mandatory may keep them away from v5, potentially 
decreasing its adoption rate.

The change is less about checksumming per-se, and more about improving protocol 
performance by making framing sane. In particular when compression is enabled. 
In addition, I strongly believe that no new protocol should be released without 
application-level checksumming built-in, and v5 is a new protocol.

bq. For these drivers, it's a race against time that starts now: they must 
implement protocol checksumming before Cassandra 4.0 GA, or they will lose 
access to all the v5 features implemented so far, especially keyspace-per-query 
– which is a nice usability improvement that many users are waiting for.

All of the v5-beta features implemented so far are nice-to-haves and 
non-essential, IMO. The new framing, to me, *is* the core of V5. And if those 
drivers don't immediately support the new protocol upon C* release, that is 
fine, too. C* and drivers are on separate release cadences, for one, and V4 
protocol is still supported, for two.

bq. Making checksumming opt-in would allow many drivers to be ready for 
Cassandra 4.0 GA, and others to catch up quickly. We feel this would greatly 
contribute to promoting Cassandra 4.0's adoption.

Again, it would be nice if all the drivers supported protocol v5 by the time C* 
4.0 was out, but I don't see it as an issue if they don't. Protocol v5 is not 
the selling feature of C* 4.0, and other changes should provide plenty of 
motivation for people to upgrade. The drivers will still be ready for 4.0 GA - 
so long as they can speak v4 - and implement v5 at some point later.

> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol 
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
>  Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it 
> introduced checksumming/CRC32 to request and response bodies. It’s an 
> important step forward, but it doesn’t cover the entire stream. In 
> particular, the message header is not covered by a checksum or a crc, which 
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of 
> computing the CRC32 on the bytes written on the wire - losing the properties 
> of the CRC32. In some cases, due to this sequencing, attempting to decompress 
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order, 
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for 
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since 
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression 
> operate on individual message bodies rather than frames of multiple complete 
> messages. In reality, this has several important additional downsides. To 
> name a couple:
> # For compression, we are getting poor compression ratios for smaller 
> messages - when operating on tiny sequences of bytes. In reality, for most 
> small requests and responses we are discarding the compressed value as it’d 
> be smaller than the uncompressed one - incurring both redundant allocations 
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages. 
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the 
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we 
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode 
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed, 
> and that we do it before native protocol v5 graduates from beta. See 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
>  and 
>

[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta

2020-05-13 Thread Benedict Elliott Smith (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106254#comment-17106254
 ] 

Benedict Elliott Smith commented on CASSANDRA-15299:


The new framing also replaces the protocol's broken approach to compression, so 
unless we maintain essentially two distinct v5 protocols we'd be talking about 
not supporting compression for these clients either.  I don't think this is an 
acceptable option?

I also think it is a bug that we have not enforced checksums until now.  Given 
their low cost, I'm strongly against making them optional without strong 
evidence of benefit and, even then, only if permitted by the cluster 
administrator.

If the only remaining option is for clients to lag the server in features, then 
I guess that's what I vote for.

I'm surprised this is considered so onerous to implement by the clients, though.

> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol 
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
>  Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it 
> introduced checksumming/CRC32 to request and response bodies. It’s an 
> important step forward, but it doesn’t cover the entire stream. In 
> particular, the message header is not covered by a checksum or a crc, which 
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of 
> computing the CRC32 on the bytes written on the wire - losing the properties 
> of the CRC32. In some cases, due to this sequencing, attempting to decompress 
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order, 
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for 
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since 
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression 
> operate on individual message bodies rather than frames of multiple complete 
> messages. In reality, this has several important additional downsides. To 
> name a couple:
> # For compression, we are getting poor compression ratios for smaller 
> messages - when operating on tiny sequences of bytes. In reality, for most 
> small requests and responses we are discarding the compressed value as it’d 
> be smaller than the uncompressed one - incurring both redundant allocations 
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages. 
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the 
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we 
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode 
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed, 
> and that we do it before native protocol v5 graduates from beta. See 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
>  and 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15805:
--
Fix Version/s: 3.11.x
   3.0.x

> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones 
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The legacy reading code ({{LegacyLayout}} and 
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly 
> the case where a range tombstone covering multiple rows interacts with a 
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
>   k int,
>   c1 text,
>   c2 text,
>   a text,
>   b set,
>   c text,
>   PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'}, 
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same 
> table or compacted together), then this will result in the inserted row being 
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this 
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said, 
> the additional code added to CASSANDRA-15789 to force merging duplicated rows 
> if they are produced _will_ end up fixing this as a consequence (assuming 
> there is no variation of this problem that leads to other visible issues than 
> duplicated rows). That said, I "think" we'd still rather fix the source of 
> the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15805) Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones interacts with collection tombstones



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15805:
--
Reviewers: Aleksey Yeschenko, Marcus Eriksson

> Potential duplicate rows on 2.X->3.X upgrade when multi-rows range tombstones 
> interacts with collection tombstones
> --
>
> Key: CASSANDRA-15805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15805
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Local/SSTable
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
>
> The legacy reading code ({{LegacyLayout}} and 
> {{UnfilteredDeserializer.OldFormatDeserializer}}) does not handle correctly 
> the case where a range tombstone covering multiple rows interacts with a 
> collection tombstone.
> A simple example of this problem is if one runs on 2.X:
> {noformat}
> CREATE TABLE t (
>   k int,
>   c1 text,
>   c2 text,
>   a text,
>   b set,
>   c text,
>   PRIMARY KEY((k), c1, c2)
> );
> // Delete all rows where c1 is 'A'
> DELETE FROM t USING TIMESTAMP 1 WHERE k = 0 AND c1 = 'A';
> // Inserts a row covered by that previous range tombstone
> INSERT INTO t(k, c1, c2, a, b, c) VALUES (0, 'A', 'X', 'foo', {'whatever'}, 
> 'bar') USING TIMESTAMP 2;
> // Delete the collection of that previously inserted row
> DELETE b FROM t USING TIMESTAMP 3 WHERE k = 0 AND c1 = 'A' and c2 = 'X';
> {noformat}
> If the following is ran on 2.X (with everything either flushed in the same 
> table or compacted together), then this will result in the inserted row being 
> duplicated (one part containing the {{a}} column, the other the {{c}} one).
> I will note that this is _not_ a duplicate of CASSANDRA-15789 and this 
> reproduce even with the fix to {{LegacyLayout}} of this ticket. That said, 
> the additional code added to CASSANDRA-15789 to force merging duplicated rows 
> if they are produced _will_ end up fixing this as a consequence (assuming 
> there is no variation of this problem that leads to other visible issues than 
> duplicated rows). That said, I "think" we'd still rather fix the source of 
> the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15533) Don't allocate unneeded MergeIterator in OnDiskToken#iterator



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106233#comment-17106233
 ] 

Benjamin Lerer commented on CASSANDRA-15533:


[~jwest] I moved the Fix version to 4.x (I hope I understood your comment 
correctly).

> Don't allocate unneeded MergeIterator in OnDiskToken#iterator 
> --
>
> Key: CASSANDRA-15533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15533
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.x
>
>
> When reviewing CASSANDRA-15392 it became apparent that the MergeIterator 
> allocated by OnDiskToken#iterator is rarely necessary and so we should avoid 
> allocating one when not needed and skip the MergeIterator pool when needed 
> because its unlikely to be sized correctly. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15533) Don't allocate unneeded MergeIterator in OnDiskToken#iterator



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15533:
---
Fix Version/s: (was: 4.0-rc)
   (was: 4.0)
   4.x

> Don't allocate unneeded MergeIterator in OnDiskToken#iterator 
> --
>
> Key: CASSANDRA-15533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15533
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.x
>
>
> When reviewing CASSANDRA-15392 it became apparent that the MergeIterator 
> allocated by OnDiskToken#iterator is rarely necessary and so we should avoid 
> allocating one when not needed and skip the MergeIterator pool when needed 
> because its unlikely to be sized correctly. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106136#comment-17106136
 ] 

Alex Petrov commented on CASSANDRA-14557:
-

bq. do you suggest starting a new thread in dev list / slack?

I'd say dev list is the best way! Btw, not saying that the change is bad, since 
most people actually do increase RF of system keyspaces as one of the first 
things they do on their clusters. It's just that such change deserves its own 
ticket and accompanying discussion, even if it ends up being +1 from everyone.

> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
>  Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right 
> now - the system_auth table for example is created with RF=1 (to take into 
> account single node setups afaict from CASSANDRA-5112), and a user can 
> further create a keyspace with RF=1 posing availability and streaming risks 
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
>  # {{default_keyspace_rf}} (default: 1) - If replication factors are not 
> specified, use this number.
>  # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from 
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
>  * Provide defaults for new keyspaces created with SimpleStrategy or 
> NetworkTopologyStrategy (CASSANDRA-14303)
>  * Make the automatic token [allocation 
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
>  interface more intuitive allowing easy use of the new token allocation 
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply 
> don’t set the setting. For backwards compatibility the default remains 1 and 
> C* would create with RF=1, and would default to current behavior of allowing 
> any RF on keyspaces.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15809:
---
Fix Version/s: 4.0-beta

> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> The JDK is specified in the groovy DSL:
> [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
>  
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]
>  
> Background 
> thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15809) ASF CI builds for JDK11



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106125#comment-17106125
 ] 

Michael Semb Wever commented on CASSANDRA-15809:


On the back of [~adejanov...@hotmail.com]'s work with ZGC benchmarking, we may 
be wanting to run and test with JDK14 as well.

> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: shylaja kokoori
>Priority: Normal
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> The JDK is specified in the groovy DSL:
> [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
>  
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]
>  
> Background 
> thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15809:
---
Description: 
ASF CI builds today only run on JDK1.8

On the Jenkins cluster JDKs from 1.4 through to 15 are available. See attached 
screenshot for naming specifics.

This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
parity to CircleCI's workflows. (There is also the question about 
testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
Cassandra-2.2 is nearing EOL.)

 

The JDK is specified in the groovy DSL:
[https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
 

 

Some examples:
 * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
both JDK1.8 and JDK11.
 ** 
[https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
 * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
 ** 
[https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
 * Jenkins Cqlshlib tests showing matrix builds:
 ** 
[https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]

 

Background 
thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E]
 

  was:
ASF CI builds today only run on JDK1.8

On the Jenkins cluster JDKs from 1.4 through to 15 are available. See attached 
screenshot for naming specifics.

This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
parity to CircleCI's workflows. (There is also the question about 
testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
Cassandra-2.2 is nearing EOL.)

 

Some examples:
 * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
both JDK1.8 and JDK11.
 ** 
[https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
 * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
 ** 
[https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
 * Jenkins Cqlshlib tests showing matrix builds:
 ** 
[https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]


> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: shylaja kokoori
>Priority: Normal
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> The JDK is specified in the groovy DSL:
> [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
>  
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]
>  
> Background 
> thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15809) ASF CI builds for JDK11



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever reassigned CASSANDRA-15809:
--

Assignee: shylaja kokoori

> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: shylaja kokoori
>Priority: Normal
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15809:
---
Attachment: Screenshot 2020-05-13 at 09.39.56.png

> ASF CI builds for JDK11
> ---
>
> Key: CASSANDRA-15809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Priority: Normal
> Attachments: Screenshot 2020-05-13 at 09.39.56.png
>
>
> ASF CI builds today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> attached screenshot for naming specifics.
> This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
> parity to CircleCI's workflows. (There is also the question about 
> testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
> Cassandra-2.2 is nearing EOL.)
>  
> Some examples:
>  * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
> both JDK1.8 and JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
>  * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
>  ** 
> [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
>  * Jenkins Cqlshlib tests showing matrix builds:
>  ** 
> [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15748) Provide ability to configure IAuditLogger



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106097#comment-17106097
 ] 

Stefan Miklosovic edited comment on CASSANDRA-15748 at 5/13/20, 8:32 AM:
-

[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
StorageServiceMBean, that patch contains another enableAuditLog method which 
can accept a map. It is there for cases somebody really wants to pass some 
parameters via map there via JMX. There is as well the original method without 
that map, because in tooling like JConsole, that method is not invokable 
anymore if it contains a map, so there is not any way to invoke it via JMX in 
JConsole, hence the original method is left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.


was (Author: stefan.miklosovic):
[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
StorageServiceMBean, that patch contains another enableAuditLog method which 
can accept a map. It is there for cases somebody really wants to pass some 
parameters via map there via JMX. There is as well the original method without 
that map, because in tooling like JConsole, that method is not invokable 
anymore if it contains a map, so there is not any way to invoke it via JMX, 
hence the original method is left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.

> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger 
> instances. There is not any way how to "inject" configuration properties from 
> cassandra.yaml into these concrete classes. This would be very handy for 
> custom IAuditLogger implementations.
>  
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15748) Provide ability to configure IAuditLogger



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106097#comment-17106097
 ] 

Stefan Miklosovic edited comment on CASSANDRA-15748 at 5/13/20, 8:31 AM:
-

[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
StorageServiceMBean, that patch contains another enableAuditLog method which 
can accept a map. It is there for cases somebody really wants to pass some 
parameters via map there via JMX. There is as well the original method without 
that map, because in tooling like JConsole, that method is not invokable 
anymore if it contains a map, so there is not any way to invoke it via JMX, 
hence the original method is left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.


was (Author: stefan.miklosovic):
[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
NodeProbe, that patch contains another enableAuditLog method which can accept a 
map. It is there for cases somebody really wants to pass some parameters via 
map there via JMX. There is as well the original method without that map, 
because in tooling like JConsole, that method is not invokable anymore if it 
contains a map, so there is not any way to invoke it via JMX, hence the 
original method is left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.

> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger 
> instances. There is not any way how to "inject" configuration properties from 
> cassandra.yaml into these concrete classes. This would be very handy for 
> custom IAuditLogger implementations.
>  
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15748) Provide ability to configure IAuditLogger



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106097#comment-17106097
 ] 

Stefan Miklosovic edited comment on CASSANDRA-15748 at 5/13/20, 8:29 AM:
-

[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
NodeProbe, that patch contains another enableAuditLog method which can accept a 
map. It is there for cases somebody really wants to pass some parameters via 
map there via JMX. There is as well the original method without that map, 
because in tooling like JConsole, that method is not invokable anymore if it 
contains a map, so there is not any way to invoke it via JMX, hence the 
original method is left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.


was (Author: stefan.miklosovic):
[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
NodeProbe, that patch contains another enableAuditLog method which can accept a 
map. It is there for cases somebody really wants to pass some parameters via 
map there. There is as well the original method with that map, because in 
tooling like JConsole, that method is not invokable anymore if it contains a 
map, so there is not any way to invoke it via JMX, hence the original method is 
left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.

> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger 
> instances. There is not any way how to "inject" configuration properties from 
> cassandra.yaml into these concrete classes. This would be very handy for 
> custom IAuditLogger implementations.
>  
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15748) Provide ability to configure IAuditLogger



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106097#comment-17106097
 ] 

Stefan Miklosovic commented on CASSANDRA-15748:
---

[~vinaykumarcse]

yes, this was done on purpose, I think I had this discussion with Marcus 
privately, the thing is that I dont know how to propagate map 
from nodetool's enableauditlogging - there is not any way to pass map from the 
command line (at least the straightforward one). If you look closely into 
NodeProbe, that patch contains another enableAuditLog method which can accept a 
map. It is there for cases somebody really wants to pass some parameters via 
map there. There is as well the original method with that map, because in 
tooling like JConsole, that method is not invokable anymore if it contains a 
map, so there is not any way to invoke it via JMX, hence the original method is 
left there for this purpose.

[~marcuse] also raised an interesting issue that it feels overall "slightly 
wrong" to have this enable / disable method there at all. One could just 
disable logging, do something bad and enable it again. Hence the question is if 
that nodetool command is actually relevant at all.

> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger 
> instances. There is not any way how to "inject" configuration properties from 
> cassandra.yaml into these concrete classes. This would be very handy for 
> custom IAuditLogger implementations.
>  
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15809) ASF CI builds for JDK11

Michael Semb Wever created CASSANDRA-15809:
--

 Summary: ASF CI builds for JDK11
 Key: CASSANDRA-15809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15809
 Project: Cassandra
  Issue Type: Task
  Components: Build, CI
Reporter: Michael Semb Wever


ASF CI builds today only run on JDK1.8

On the Jenkins cluster JDKs from 1.4 through to 15 are available. See attached 
screenshot for naming specifics.

This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for 
parity to CircleCI's workflows. (There is also the question about 
testing/running on Cassandra-2.2 which needs to support JDK1.7, though 
Cassandra-2.2 is nearing EOL.)

 

Some examples:
 * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with 
both JDK1.8 and JDK11.
 ** 
[https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8]
 * CircleCI JDK11 workflow example. This builds and tests only with JDK11.
 ** 
[https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306]
 * Jenkins Cqlshlib tests showing matrix builds:
 ** 
[https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105653#comment-17105653
 ] 

Alex Petrov edited comment on CASSANDRA-15778 at 5/13/20, 8:13 AM:
---

I've taken a short look and wanted to precaution against ignoring or even 
purging values from the hidden compact column, since there are scenarios when 
it can contain legitimate data. 

I've been able to reproduce an issue with a similar, albeit not exactly same 
stacktrace (leaving only the part from sstable iterator downward):

{code}
ERROR 18:08:11 Uncaught exception on thread Thread[SharedPool-Worker-2,10,main]
java.lang.RuntimeException: org.apache.cassandra.serializers.MarshalException: 
EmptyType only accept empty values
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
 ~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_171]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
 ~[main/:na]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
 [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
[main/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
Caused by: org.apache.cassandra.serializers.MarshalException: EmptyType only 
accept empty values
at 
org.apache.cassandra.serializers.EmptySerializer.validate(EmptySerializer.java:42)
 ~[main/:na]
at 
org.apache.cassandra.db.marshal.AbstractType.validate(AbstractType.java:159) 
~[main/:na]
at 
org.apache.cassandra.db.marshal.AbstractType.validateIfFixedSize(AbstractType.java:390)
 ~[main/:na]
at 
org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:1457)
 ~[main/:na]
at 
org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:1377)
 ~[main/:na]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:542)
 ~[main/:na]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:519)
 ~[main/:na]
at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:336)
 ~[main/:na]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:140)
 ~[main/:na]
at 
org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:172)
 ~[main/:na]
at 
org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:336)
 ~
{code}

In short, this is a table created on 2.2 side with: {{CREATE TABLE table_0 (key 
text, value text, PRIMARY KEY (key, value)) WITH COMPACT STORAGE;}}

If you write data into this table with thrift ({{ThriftClient#batch_mutate}}), 
you will end up with data in the hidden column, which looks like this on the 
2.1 side:

{code}
select * from ks1.table_0;

 key   | value
---+---
 key22 |  key1
 key22 |  key4
 key11 |  key1
 key11 |  key4

(4 rows)
{code}

And like this in sstabledump:

{code}
[
{"key": "key22",
 "cells": [["key1","76616c756531",1589306163593],
   ["key4","76616c756534",1589306163594]]},
{"key": "key11",
 "cells": [["key1","76616c756531",1589306163593],
   ["key4","76616c756534",1589306163594]]}
]
{code}

Of course, on 3.x side you wouldn't be able to see the values.

UPDATE: Was able to reproduce it: 

{code}
java.lang.RuntimeException: 
org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
/Users/ifesdjeen/foss/java/apache-cassandra/data/data/ks1/table_0-05609b80948511eabfa891431c623cd5/md-2-big-Data.db
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
 ~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_171]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
 ~[main/:na]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
 [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
[main/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
/Users/ifesdjeen/foss/java/apache-cassandra/data/data/ks1/table_0-05609b80948511eabfa891431c623cd5/md-2-big-Data.db
at

[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options

2020-05-13 Thread Sumanth Pasupuleti (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106066#comment-17106066
 ] 

Sumanth Pasupuleti commented on CASSANDRA-14557:


Thanks for the rebase and kicking off CI runs.

I agree, it may be non-intuitive that this feature also applies to system 
keyspaces, and not just user keyspaces. Infact, one of the primary reasons why 
I felt this feature to be useful is to make sure system_auth keyspace comes up 
with an RF of 3, vs the default 1.
As for the discussion, do you suggest starting a new thread in dev list / slack?

Will take a look at the dtests and update the ticket.

Also, I've noted that, its best to change the state to "ready to commit" after 
the ticket is reviewed, thanks for the suggestion.



> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
>  Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right 
> now - the system_auth table for example is created with RF=1 (to take into 
> account single node setups afaict from CASSANDRA-5112), and a user can 
> further create a keyspace with RF=1 posing availability and streaming risks 
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
>  # {{default_keyspace_rf}} (default: 1) - If replication factors are not 
> specified, use this number.
>  # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from 
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
>  * Provide defaults for new keyspaces created with SimpleStrategy or 
> NetworkTopologyStrategy (CASSANDRA-14303)
>  * Make the automatic token [allocation 
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
>  interface more intuitive allowing easy use of the new token allocation 
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply 
> don’t set the setting. For backwards compatibility the default remains 1 and 
> C* would create with RF=1, and would default to current behavior of allowing 
> any RF on keyspaces.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15748) Provide ability to configure IAuditLogger

2020-05-13 Thread Vinay Chella (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106065#comment-17106065
 ] 

Vinay Chella commented on CASSANDRA-15748:
--

This patch looks good to me except 1 minor nit and question related to nodetool 
command support. 

Were the changes for the NodeTool command in `EnableAuditLog.java` 
intentionally not included to support the new class and parameter format? If we 
are providing the ability to configure custom class with parameters, it is good 
to support the same via nodetool as well. If we decide not to support, calling 
it out could be a good idea


> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger 
> instances. There is not any way how to "inject" configuration properties from 
> cassandra.yaml into these concrete classes. This would be very handy for 
> custom IAuditLogger implementations.
>  
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637

2020-05-13 Thread Eduard Tudenhoefner (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106061#comment-17106061
 ] 

Eduard Tudenhoefner commented on CASSANDRA-15776:
-

sorry, wasn't aware you're waiting for my review because I assumed that 2 
reviews would be enough. Changes LGTM

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads