[jira] [Comment Edited] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186245#comment-17186245
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16063 at 8/28/20, 4:26 AM:
---

These are the four branches I worked on for this patch:

[C* 
3.0|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.0] 
| [C* 
3.11|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.11]
 | [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063] | 
[DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-16063]
 

4 things have been done:
 1) Check SSTables for latest version before dropping compact storage commits - 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c]
 and 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/c0c43e90644b28b9b363fa7aba55adbf95dd5bd7]
 2) Allow skipping commit log replay was not failing on descriptor errors, this 
was corrected on all branches in order to support our strategy from the next 
point.
 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/dd6207e6639576a8090ea5930f46c3cf2a4a5971]
 | 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/ad6f1f9d5106b07a001a847a63bb4b3068b0c599]
 | 
[trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/62affe6e9e5dc654ba729a97d136e16c37f392fd]
 3) Move compact storage validation earlier in startup process with detailed 
[instructions|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72#diff-2220885f2835194f87cce0d27ac87c73R915]
 to the users 
[here|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72]

With this change we no longer write out sstable data (only a CL segment).

4) Two upgrade test created 
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/commits/CASSANDRA-16063]
 Trunk CI run: 

[Java 
8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/324e7e64-097e-4cb3-8521-d5c4eacaca9c]
 and [Java 
11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/559d91f6-d80a-4eb9-8211-7220db8618a5]
 CI runs did not show anything disturbing. The failures presented look related 
to the latest timeouts and incremental repair failures which are tackled by the 
community this week. I will check tomorrow whether any of them requires a new 
ticket to be raised. 

[~slebresne] do you mind to make a review, please? Also, I need the upgrade 
tests to be pushed in Jenkins as they did not appear in CircleCI and I have 
some troubles to run them locally. Do you mind to help me with that too? (I 
have no access there)

_About the order of commits, I would say:_
 _1st the tests; Then 1) -> 2) -> 3)_


was (Author: e.dimitrova):
 These are the four branches I worked on for this patch:

[C* 
3.0|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.0] 
| [C* 
3.11|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.11]
 | [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063] | 
[DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-16063]
 

4 things have been done:
 1) Check SSTables for latest version before dropping compact storage commits - 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c]
 and 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/c0c43e90644b28b9b363fa7aba55adbf95dd5bd7]
 2) Allow skipping commit log replay was not failing on descriptor errors, this 
was corrected on all branches in order to support our strategy from the next 
point.
 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/dd6207e6639576a8090ea5930f46c3cf2a4a5971]
 | 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/ad6f1f9d5106b07a001a847a63bb4b3068b0c599]
 | 
[trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/62affe6e9e5dc654ba729a97d136e16c37f392fd]
 3) Move compact storage validation earlier in startup process with detailed 
[instructions|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72#diff-2220885f2835194f87cce0d27ac87c73R915]
 to the users 
[here|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72]

With this change we no longer write out sstable data (only a CL segment).

4) Two upgrade test created 
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/commits/CASSANDRA-16063]
Trunk CI run: 

[Java 
8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/324e7e64-097e-4cb3-8521-d5c4eacaca9c]
 and [Java 

[jira] [Comment Edited] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186245#comment-17186245
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16063 at 8/28/20, 4:25 AM:
---

 These are the four branches I worked on for this patch:

[C* 
3.0|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.0] 
| [C* 
3.11|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.11]
 | [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063] | 
[DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-16063]
 

4 things have been done:
 1) Check SSTables for latest version before dropping compact storage commits - 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c]
 and 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/c0c43e90644b28b9b363fa7aba55adbf95dd5bd7]
 2) Allow skipping commit log replay was not failing on descriptor errors, this 
was corrected on all branches in order to support our strategy from the next 
point.
 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/dd6207e6639576a8090ea5930f46c3cf2a4a5971]
 | 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/ad6f1f9d5106b07a001a847a63bb4b3068b0c599]
 | 
[trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/62affe6e9e5dc654ba729a97d136e16c37f392fd]
 3) Move compact storage validation earlier in startup process with detailed 
[instructions|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72#diff-2220885f2835194f87cce0d27ac87c73R915]
 to the users 
[here|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72]

With this change we no longer write out sstable data (only a CL segment).

4) Two upgrade test created 
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/commits/CASSANDRA-16063]
Trunk CI run: 

[Java 
8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/324e7e64-097e-4cb3-8521-d5c4eacaca9c]
 and [Java 
11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/559d91f6-d80a-4eb9-8211-7220db8618a5]
 CI runs did not show anything disturbing. The failures presented look related 
to the latest timeouts and incremental repair failures which are tackled by the 
community this week. I will check tomorrow whether any of them requires a new 
ticket to be raised. 

[~slebresne] do you mind to make a review, please? Also, I need the upgrade 
tests to be pushed in Jenkins as they did not appear in CircleCI and I have 
some troubles to run them locally. Do you mind to help me with that too? (I 
have no access there)

_About the order of commits, I would say:
1st the tests; Then 1) ->  2) -> 3)_


was (Author: e.dimitrova):
 These are the four branches I worked on for this patch:

[C* 
3.0|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.0] 
| [C* 
3.11|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.11]
 | [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063] | 
[DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-16063]
 

4 things have been done:
 1) Check SSTables for latest version before dropping compact storage commits - 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c]
 and 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/c0c43e90644b28b9b363fa7aba55adbf95dd5bd7]
 2) Allow skipping commit log replay was not failing on descriptor errors, this 
was corrected on all branches in order to support our strategy from the next 
point.
 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/dd6207e6639576a8090ea5930f46c3cf2a4a5971]
 | 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/ad6f1f9d5106b07a001a847a63bb4b3068b0c599]
 | 
[trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/62affe6e9e5dc654ba729a97d136e16c37f392fd]
 3) Move compact storage validation earlier in startup process with detailed 
[instructions|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72#diff-2220885f2835194f87cce0d27ac87c73R915]
 to the users 
[here|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72]

With this change we no longer write out sstable data (only a CL segment).

4) Two upgrade test created 
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/commits/CASSANDRA-16063]
Trunk CI run: 

[Java 
8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/324e7e64-097e-4cb3-8521-d5c4eacaca9c]
 and [Java 

[jira] [Updated] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16063:

Test and Documentation Plan: 
https://issues.apache.org/jira/browse/CASSANDRA-16063?focusedCommentId=17186245=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17186245
 Status: Patch Available  (was: In Progress)

> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The code to handle compact tables has been removed from 4.0, and the intended 
> upgrade path to 4.0 for users having compact tables on 3.x is that they must 
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table) 
> and may try upgrading despite still having compact tables. If they do so, the 
> intent is that the node will _not_ start, with a message clearly indicating 
> the pre-upgrade step the user has missed. The user will then downgrade back 
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and 
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables 
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within 
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
>   But by then, we've _at least_ called 
> {{SystemKeyspace.persistLocalMetadata()}}} and 
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, 
> and even possibly flush new {{na}} format sstables. As a results, a user 
> might not be able to seemlessly restart the node on 3.x (to drop compact 
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0 
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0 
> with compact storage, you can downgrade back with no intervention whatsoever).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186245#comment-17186245
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16063:
-

 These are the four branches I worked on for this patch:

[C* 
3.0|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.0] 
| [C* 
3.11|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063-3.11]
 | [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-16063] | 
[DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-16063]
 

4 things have been done:
 1) Check SSTables for latest version before dropping compact storage commits - 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c]
 and 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/c0c43e90644b28b9b363fa7aba55adbf95dd5bd7]
 2) Allow skipping commit log replay was not failing on descriptor errors, this 
was corrected on all branches in order to support our strategy from the next 
point.
 
[3.0|https://github.com/ekaterinadimitrova2/cassandra/commit/dd6207e6639576a8090ea5930f46c3cf2a4a5971]
 | 
[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/ad6f1f9d5106b07a001a847a63bb4b3068b0c599]
 | 
[trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/62affe6e9e5dc654ba729a97d136e16c37f392fd]
 3) Move compact storage validation earlier in startup process with detailed 
[instructions|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72#diff-2220885f2835194f87cce0d27ac87c73R915]
 to the users 
[here|https://github.com/ekaterinadimitrova2/cassandra/commit/c4835158f2815ab90bf6fdb907e95861984f2c72]

With this change we no longer write out sstable data (only a CL segment).

4) Two upgrade test created 
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/commits/CASSANDRA-16063]
Trunk CI run: 

[Java 
8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/324e7e64-097e-4cb3-8521-d5c4eacaca9c]
 and [Java 
11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/338/workflows/559d91f6-d80a-4eb9-8211-7220db8618a5]
 CI runs did not show anything disturbing. The failures presented look related 
to the latest timeouts and incremental repair failures which are tackled by the 
community this week. I will check tomorrow whether any of them requires a new 
ticket to be raised. 

[~slebresne] do you mind to make a review, please? Also, I need the upgrade 
tests to be pushed in Jenkins as they did not appear in CircleCI and I have 
some troubles to run them locally. Do you mind to help me with that too? (I 
have no access there)

> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The code to handle compact tables has been removed from 4.0, and the intended 
> upgrade path to 4.0 for users having compact tables on 3.x is that they must 
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table) 
> and may try upgrading despite still having compact tables. If they do so, the 
> intent is that the node will _not_ start, with a message clearly indicating 
> the pre-upgrade step the user has missed. The user will then downgrade back 
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and 
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables 
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within 
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
>   But by then, we've _at least_ called 
> {{SystemKeyspace.persistLocalMetadata()}}} and 
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, 
> and even possibly flush new {{na}} format sstables. As a results, a user 
> might not be able to seemlessly restart the node on 3.x (to drop compact 
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0 
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0 
> with compact storage, you 

[jira] [Comment Edited] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186122#comment-17186122
 ] 

Caleb Rackliffe edited comment on CASSANDRA-15909 at 8/28/20, 4:12 AM:
---

I've made changes following my own review suggestions and posted a new, rebased 
version of the patch.

[trunk|https://github.com/apache/cassandra/pull/728], [j8 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/99/workflows/72c69ea8-f347-4b00-aed8-bd465f3549ff],
 [j11 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/99/workflows/fdac8d5b-2f3d-46f9-8e67-a5a9ed8ab966]

I think the {{ConnectionTest}} failure is a pre-existing issue.


was (Author: maedhroz):
I've made changes following my own review suggestions and posted a new, rebased 
version of the patch.

[trunk|https://github.com/apache/cassandra/pull/728], [j8 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/d86b8627-c602-44e1-b6d5-cedaa67bc5b8],
 [j11 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/70f8c126-5d86-4e8a-a346-726beb45b87b]

I think the {{ConnectionTest}} failure is a pre-existing issue.

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15393) Add byte array backed cells

[Josh McKenzie (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186177#comment-17186177
 ] 

Josh McKenzie edited comment on CASSANDRA-15393 at 8/28/20, 12:14 AM:
--

I think it was the "perf improvements are acceptable up until release as per 
the agreed upon release lifecycle" so no reason to block beta1 on it. We can 
discuss on ML.


was (Author: jmckenzie):
I think it was the "perf improvements are allowed up until release as per the 
agreed upon release lifecycle" so no reason to block beta1 on it. We can 
discuss on ML.

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells

[Josh McKenzie (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186177#comment-17186177
 ] 

Josh McKenzie commented on CASSANDRA-15393:
---

I think it was the "perf improvements are allowed up until release as per the 
agreed upon release lifecycle" so no reason to block beta1 on it. We can 
discuss on ML.

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15909:
--
Reviewers: Caleb Rackliffe, David Capwell, David Capwell  (was: Caleb 
Rackliffe, David Capwell)
   Caleb Rackliffe, David Capwell, David Capwell  (was: Caleb 
Rackliffe, David Capwell)
   Status: Review In Progress  (was: Patch Available)

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15988) Add nodetool getfullquerylog

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15988:
--
Reviewers: David Capwell, David Capwell  (was: David Capwell)
   David Capwell, David Capwell
   Status: Review In Progress  (was: Patch Available)

left feedback in PR

> Add nodetool getfullquerylog 
> -
>
> Key: CASSANDRA-15988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15988
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tool/fql
>Reporter: Ekaterina Dimitrova
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This ticket is raised based on CASSANDRA-15791 and valuable feedback provided 
> by [~jshook].
> There are two outstanding questions:
>  * forming the exact shape of such a command and how it can benefit the 
> users; to be discussed in detail in this ticket
>  * Is this a thing we as a project can add to 4.0 beta or it should be 
> considered in 4.1 for example
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186122#comment-17186122
 ] 

Caleb Rackliffe edited comment on CASSANDRA-15909 at 8/27/20, 10:01 PM:


I've made changes following my own review suggestions and posted a new, rebased 
version of the patch.

[trunk|https://github.com/apache/cassandra/pull/728], [j8 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/d86b8627-c602-44e1-b6d5-cedaa67bc5b8],
 [j11 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/70f8c126-5d86-4e8a-a346-726beb45b87b]

I think the {{ConnectionTest}} failure is a pre-existing issue.


was (Author: maedhroz):
I've made changes following my own review suggestions and posted a new, rebased 
version of the patch.

[trunk|https://github.com/apache/cassandra/pull/728], [j8 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/d86b8627-c602-44e1-b6d5-cedaa67bc5b8],
 [j11 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/70f8c126-5d86-4e8a-a346-726beb45b87b]

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15909:

Reviewers: Caleb Rackliffe, David Capwell  (was: Caleb Rackliffe)

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15971) full query log needs improvement

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15971:
--
  Fix Version/s: 4.0-beta2
Source Control Link: 
https://github.com/apache/cassandra/commit/3a2d709b9c782e3e2f163a46341aed073e7a6b0d
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> full query log needs improvement
> 
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta2
>
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench 
> usage, I ran across many issues which would make it painful for users. Since 
> there were several, they will be added to a single issue for now. This issue 
> can be broken up if needed.
> 
> FQL doesn't work on my system, even though it says it is logging queries. 
> With the following configuration in cassandra.yaml:
>  
> {noformat}
> full_query_logging_options:
>     log_dir: /REDACTED/fullquerylogs
>     roll_cycle: HOURLY
>     block: true
>     max_queue_weight: 268435456 # 256 MiB
>     max_log_size: 17179869184 # 16 GiB
>     ## archive command is "/path/to/script.sh %path" where %path is replaced 
> with the file being rolled:
>     # archive_command:
>     # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a 
> single file `directory-listing.cq4t` is created, which is a 64K sized file of 
> zeroes.
>  
> 
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>  
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long) 
> -- StackTrace -- 
> java.lang.NoSuchMethodException: 
> sun.nio.ch.FileChannelImpl.map0(int,long,long) 
>     at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553) 
>     at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51) 
>     at 
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>  
> Subsequent calls produce normal output:
>  
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog 
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs 
> See 'nodetool help' or 'nodetool help '.{noformat}
>  
> 
> nodetool missing getfullquerylog makes it difficult to verify current 
> fullquerylog state without changing it. The conventions for nodetool commands 
> should be followed to avoid confusing users.
> 
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating 
> the default The most commonly used fqltool commands are..
> 
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is 
> malformatted, mixing the appearance of configuration with comments, which is 
> confusing at best.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15971) full query log needs improvement

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15971:
--
Status: Ready to Commit  (was: Changes Suggested)

> full query log needs improvement
> 
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
>  Labels: pull-request-available
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench 
> usage, I ran across many issues which would make it painful for users. Since 
> there were several, they will be added to a single issue for now. This issue 
> can be broken up if needed.
> 
> FQL doesn't work on my system, even though it says it is logging queries. 
> With the following configuration in cassandra.yaml:
>  
> {noformat}
> full_query_logging_options:
>     log_dir: /REDACTED/fullquerylogs
>     roll_cycle: HOURLY
>     block: true
>     max_queue_weight: 268435456 # 256 MiB
>     max_log_size: 17179869184 # 16 GiB
>     ## archive command is "/path/to/script.sh %path" where %path is replaced 
> with the file being rolled:
>     # archive_command:
>     # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a 
> single file `directory-listing.cq4t` is created, which is a 64K sized file of 
> zeroes.
>  
> 
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>  
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long) 
> -- StackTrace -- 
> java.lang.NoSuchMethodException: 
> sun.nio.ch.FileChannelImpl.map0(int,long,long) 
>     at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553) 
>     at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51) 
>     at 
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>  
> Subsequent calls produce normal output:
>  
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog 
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs 
> See 'nodetool help' or 'nodetool help '.{noformat}
>  
> 
> nodetool missing getfullquerylog makes it difficult to verify current 
> fullquerylog state without changing it. The conventions for nodetool commands 
> should be followed to avoid confusing users.
> 
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating 
> the default The most commonly used fqltool commands are..
> 
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is 
> malformatted, mixing the appearance of configuration with comments, which is 
> confusing at best.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Improve FQL doc page

[dcapwell]

This is an automated email from the ASF dual-hosted git repository.

dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 3a2d709  Improve FQL doc page
3a2d709 is described below

commit 3a2d709b9c782e3e2f163a46341aed073e7a6b0d
Author: Lorina Poland 
AuthorDate: Thu Aug 27 14:31:03 2020 -0700

Improve FQL doc page

patch by Lorina Poland; reviewed by David Cappwell, Ekaterina Dimitrova for 
CASSANDRA-15971
---
 doc/source/new/fqllogging.rst | 676 ++
 1 file changed, 288 insertions(+), 388 deletions(-)

diff --git a/doc/source/new/fqllogging.rst b/doc/source/new/fqllogging.rst
index 08de1c7..5a78931 100644
--- a/doc/source/new/fqllogging.rst
+++ b/doc/source/new/fqllogging.rst
@@ -14,174 +14,170 @@
 .. See the License for the specific language governing permissions and
 .. limitations under the License.
 
-Full Query Logging
---
+Full Query Logging (FQL)
+
 
-Apache Cassandra 4.0 adds a new feature to support a means of logging all 
queries as they were invoked (`CASSANDRA-13983
-`_). For correctness 
testing it's useful to be able to capture production traffic so that it can be 
replayed against both the old and new versions of Cassandra while comparing the 
results.
+Apache Cassandra 4.0 adds a new highly performant feature that supports live 
query logging (`CASSANDRA-13983 
`_). 
+FQL is safe for production use, with configurable limits to heap memory and 
disk space to prevent out-of-memory errors.
+This feature is useful for live traffic capture, as well as traffic replay. 
The tool provided can be used for both debugging query traffic and migration.
+New ``nodetool`` options are also added to enable, disable or reset FQL, as 
well as a new tool to read and replay the binary logs.
+The full query logging (FQL) capability uses `Chronicle-Queue 
`_ to rotate a log of queries. 
+Full query logs will be referred to as *logs* for the remainder of the page.
 
-Cassandra 4.0 includes an implementation of a full query logging (FQL) that 
uses chronicle-queue to implement a rotating log of queries. Some of the 
features of FQL are:
+Some of the features of FQL are:
 
-- Single thread asynchronously writes log entries to disk to reduce impact on 
query latency
-- Heap memory usage bounded by a weighted queue with configurable maximum 
weight sitting in front of logging thread
-- If the weighted queue is full producers can be blocked or samples can be 
dropped
-- Disk utilization is bounded by deleting old log segments once a configurable 
size is reached
-- The on disk serialization uses a flexible schema binary format 
(chronicle-wire) making it easy to skip unrecognized fields, add new ones, and 
omit old ones.
-- Can be enabled and configured via JMX, disabled, and reset (delete on disk 
data), logging path is configurable via both JMX and YAML
-- Introduce new ``fqltool`` in ``/bin`` that currently implements ``Dump`` 
which can dump in a readable format full query logs as well as follow active 
full query logs. FQL ``Replay`` and ``Compare`` are also available.
+- The impact on query latency is reduced by asynchronous single-thread log 
entry writes to disk.
+- Heap memory usage is bounded by a weighted queue, with configurable maximum 
weight sitting in front of logging thread.
+- If the weighted queue is full, producers can be blocked or samples can be 
dropped.
+- Disk utilization is bounded by a configurable size, deleting old log 
segments once the limit is reached.
+- A flexible schema binary format, `Chronicle-Wire 
`_, for on-disk serialization that 
can skip unrecognized fields, add new ones, and omit old ones.
+- Can be enabled, disabled, or reset (to delete on-disk data) using the JMX 
tool, ``nodetool``.
+- Can configure the settings in either the `cassandra.yaml` file or by using 
``nodetool``.
+- Introduces new ``fqltool`` that currently can ``Dump`` the binary logs to a 
readable format. Other options are ``Replay`` and ``Compare``.
 
-Cassandra 4.0 has a binary full query log based on Chronicle Queue that can be 
controlled using ``nodetool enablefullquerylog``, ``disablefullquerylog``, and 
``resetfullquerylog``. The log contains all queries invoked, approximate time 
they were invoked, any parameters necessary to bind wildcard values, and all 
query options. A readable version of the log can be dumped or tailed using the 
new ``bin/fqltool`` utility. The full query log is designed to be safe to use 
in production and limi [...]
+FQL logs all successful Cassandra Query Language (CQL) requests, both events 
that modify the data and those that query. 
+While audit logs also 

[jira] [Updated] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15909:

Status: Patch Available  (was: In Progress)

I've made changes following my own review suggestions and posted a new, rebased 
version of the patch.

[trunk|https://github.com/apache/cassandra/pull/728], [j8 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/d86b8627-c602-44e1-b6d5-cedaa67bc5b8],
 [j11 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/98/workflows/70f8c126-5d86-4e8a-a346-726beb45b87b]

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15909:

Status: In Progress  (was: Changes Suggested)

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16077) Only allow strings to be passed to JMX authentication

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16077:
-
Test and Documentation Plan: none needed
 Status: Patch Available  (was: Open)

> Only allow strings to be passed to JMX authentication
> -
>
> Key: CASSANDRA-16077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16077
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/JMX
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.8, 4.0-beta2, 2.2.18, 2.1.22
>
>
> It doesn't make sense to allow other object types.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16077) Only allow strings to be passed to JMX authentication

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186118#comment-17186118
 ] 

Brandon Williams commented on CASSANDRA-16077:
--

CI:
* trunk
** https://ci-cassandra.apache.org/job/Cassandra-devbranch/285/
* 3.11
** https://ci-cassandra.apache.org/job/Cassandra-devbranch/284/
* 3.0
** https://ci-cassandra.apache.org/job/Cassandra-devbranch/289/
* 2.2
** https://ci-cassandra.apache.org/job/Cassandra-devbranch/288/

> Only allow strings to be passed to JMX authentication
> -
>
> Key: CASSANDRA-16077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16077
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/JMX
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.8, 4.0-beta2, 2.2.18, 2.1.22
>
>
> It doesn't make sense to allow other object types.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging

[Adam Holmberg (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186105#comment-17186105
 ] 

Adam Holmberg commented on CASSANDRA-15958:
---

Sorry I wasn't more precise on the third option. There, I was suggesting that 
we stop trying to calculate the earliest possible prune, and just make it prune 
periodically -- say every 10ms. Then there's no deadline to adjust forward or 
backward and synchronize relative to new messages coming in -- it's just a 
periodic task.

>Therefore, it seems a special case in the test. It should not harm in 
>production.
It sound like in this paragraph you're agreeing with my option #1. If that is 
the case (and we don't hear any opposition), when I get back to it I'll update 
the test to be robust against this.

> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build: 
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build: 
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> java.util.concurrent.TimeoutException
>   at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258)
>   at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143)
>   at 
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268)
>   at 
> org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236)
>   at 
> org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16079) Improve dtest runtime

[Adam Holmberg (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186072#comment-17186072
 ] 

Adam Holmberg edited comment on CASSANDRA-16079 at 8/27/20, 7:51 PM:
-

One idea that was relayed (with attribution to [~mck]] and [~blambov]):

{quote}remove the bootstrapping from all dtests that don't need it. e.g. to 
bootstrap parameterized ccm clusters and save them as templates, and then have 
each dtest to copy a ccm template and start it up. that deduces sequential 
bootstrap time with parallel startup time, on basically every dtest.{quote}


was (Author: aholmber):
One idea that was relayed (with attribution to [~mck2] and [~blambov]):

{quote}remove the bootstrapping from all dtests that don't need it. e.g. to 
bootstrap parameterized ccm clusters and save them as templates, and then have 
each dtest to copy a ccm template and start it up. that deduces sequential 
bootstrap time with parallel startup time, on basically every dtest.{quote}

> Improve dtest runtime
> -
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
> [30% increase in run 
> time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. 
> While that change was accepted, we wanted to spin out a ticket to optimize 
> dtests in an attempt to gain back some of that runtime.
> At this time we don't have concrete improvements in mind, so the first order 
> of this ticket will be to analyze the state of things currently, and try to 
> ascertain some valuable optimizations. Once the problems are understood, we 
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16079) Improve dtest runtime

[Adam Holmberg (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186072#comment-17186072
 ] 

Adam Holmberg edited comment on CASSANDRA-16079 at 8/27/20, 7:51 PM:
-

One idea that was relayed (with attribution to [~mck]] and [~blambov]):

{quote}remove the bootstrapping from all dtests that don't need it. e.g. to 
bootstrap parameterized ccm clusters and save them as templates, and then have 
each dtest to copy a ccm template and start it up. that deduces sequential 
bootstrap time with parallel startup time, on basically every dtest.{quote}


was (Author: aholmber):
One idea that was relayed (with attribution to [~mck]] and [~blambov]):

{quote}remove the bootstrapping from all dtests that don't need it. e.g. to 
bootstrap parameterized ccm clusters and save them as templates, and then have 
each dtest to copy a ccm template and start it up. that deduces sequential 
bootstrap time with parallel startup time, on basically every dtest.{quote}

> Improve dtest runtime
> -
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
> [30% increase in run 
> time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. 
> While that change was accepted, we wanted to spin out a ticket to optimize 
> dtests in an attempt to gain back some of that runtime.
> At this time we don't have concrete improvements in mind, so the first order 
> of this ticket will be to analyze the state of things currently, and try to 
> ascertain some valuable optimizations. Once the problems are understood, we 
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16074) Add metric for client concurrent byte throttle

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16074:
--
Status: Changes Suggested  (was: Review In Progress)

> Add metric for client concurrent byte throttle
> --
>
> Key: CASSANDRA-16074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16074
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Messaging/Client, Observability/Metrics
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
>
> Add a metric to expose the current bytes and bytes per ip used that is used 
> in the existing throttle so its possible to determine what to set it to.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16079) Improve dtest runtime

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186075#comment-17186075
 ] 

Brandon Williams commented on CASSANDRA-16079:
--

That is a really great idea, I don't think we have all that many combinations 
of options for bootstrapping in the dtests currently.

> Improve dtest runtime
> -
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
> [30% increase in run 
> time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. 
> While that change was accepted, we wanted to spin out a ticket to optimize 
> dtests in an attempt to gain back some of that runtime.
> At this time we don't have concrete improvements in mind, so the first order 
> of this ticket will be to analyze the state of things currently, and try to 
> ascertain some valuable optimizations. Once the problems are understood, we 
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16074) Add metric for client concurrent byte throttle

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16074:
--
Reviewers: David Capwell, Michael Semb Wever, David Capwell  (was: David 
Capwell, Michael Semb Wever)
   David Capwell, Michael Semb Wever, David Capwell  (was: Michael 
Semb Wever)
   Status: Review In Progress  (was: Patch Available)

Gave feedback in PR

> Add metric for client concurrent byte throttle
> --
>
> Key: CASSANDRA-16074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16074
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Messaging/Client, Observability/Metrics
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
>
> Add a metric to expose the current bytes and bytes per ip used that is used 
> in the existing throttle so its possible to determine what to set it to.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16079) Improve dtest runtime

[Adam Holmberg (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Holmberg updated CASSANDRA-16079:
--
Description: 
A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
[30% increase in run 
time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. 
While that change was accepted, we wanted to spin out a ticket to optimize 
dtests in an attempt to gain back some of that runtime.

At this time we don't have concrete improvements in mind, so the first order of 
this ticket will be to analyze the state of things currently, and try to 
ascertain some valuable optimizations. Once the problems are understood, we 
will break down subtasks to divide the work.

Some areas to consider:
* cluster reuse
* C* startup optimizations
* Tests that should be ported to in-JVM dtest or even unit tests

  was:
A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
30% increase in run time. While that change was accepted, we wanted to spin out 
a ticket to optimize dtests in an attempt to gain back some of that runtime.

At this time we don't have concrete improvements in mind, so the first order of 
this ticket will be to analyze the state of things currently, and try to 
ascertain some valuable optimizations. Once the problems are understood, we 
will break down subtasks to divide the work.

Some areas to consider:
* cluster reuse
* C* startup optimizations
* Tests that should be ported to in-JVM dtest or even unit tests


> Improve dtest runtime
> -
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
> [30% increase in run 
> time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. 
> While that change was accepted, we wanted to spin out a ticket to optimize 
> dtests in an attempt to gain back some of that runtime.
> At this time we don't have concrete improvements in mind, so the first order 
> of this ticket will be to analyze the state of things currently, and try to 
> ascertain some valuable optimizations. Once the problems are understood, we 
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16079) Improve dtest runtime

[Adam Holmberg (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186072#comment-17186072
 ] 

Adam Holmberg commented on CASSANDRA-16079:
---

One idea that was relayed (with attribution to [~mck2] and [~blambov]):

{quote}remove the bootstrapping from all dtests that don't need it. e.g. to 
bootstrap parameterized ccm clusters and save them as templates, and then have 
each dtest to copy a ccm template and start it up. that deduces sequential 
bootstrap time with parallel startup time, on basically every dtest.{quote}

> Improve dtest runtime
> -
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
> 30% increase in run time. While that change was accepted, we wanted to spin 
> out a ticket to optimize dtests in an attempt to gain back some of that 
> runtime.
> At this time we don't have concrete improvements in mind, so the first order 
> of this ticket will be to analyze the state of things currently, and try to 
> ascertain some valuable optimizations. Once the problems are understood, we 
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16066) Update and rework cassandra-website material to work with Antora

[Lorina Poland (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lorina Poland updated CASSANDRA-16066:
--
Description: 
*We want to modernise the way the website is built*
 Rework the cassandra-website repository to generate a UI bundle containing 
resources that Antora will use when generating the Cassandra documents and 
website.

*The existing method is starting to become dated*
 The documentation and website templates are currently in markdown format. 
Sphinx is used to generate the Cassandra documentation and Jekyll generates the 
website content. One of the major issues with the existing website tooling is 
that the live preview server (render site as it is being updated) is really 
slow. There is a preview server that is really fast, however it is unable to 
detect changes to the content and render automatically.

*We are migrating the docs to be rendered with Antora*
 The work in CASSANDRA-16029 is converting the document templates to AsciiDoc 
format. Sphinx is being replaced by Antora to generate the documentation 
content. This change has two advantages:
 * More flexibility if the Apache Cassandra documentation look and feel needs 
to be updated or redesigned.
 * More modern look and feel to the documentation.

*We can use Antora to generate the website as well*
 Antora could also be used to generate the Cassandra website content. As 
suggested on the [mailing 
list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15577.html] this 
would require the existing markdown templates to be converted to AsciiDoc as 
well.

*Antora needs a UI bundle to style content*
 For Antora to generate the document content and potentially the website 
content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the 
HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and 
(site-wide) images. As such, it provides both the visual theme and user 
interactions for the documentation. Effectively the UI bundle is the templates 
and styling that are applied to the documentation and website content.

*The [cassandra-website|https://github.com/apache/cassandra-website] repository 
can be used to generate the UI bundle*
 All the resources associated with templating and styling the documentation and 
website can be placed in the 
[cassandra-website|https://github.com/apache/cassandra-website] repository. In 
this case the repository would serve two purposes;
 * Generation of the UI bundle resources.
 * Serve the production website content.

*The [cassandra|https://github.com/apache/cassandra] repository would contain 
the documentation material, while the rest of the non-versioned pages would 
contain that material*
 * All other material that is used to generate documentation would live in the 
[cassandra|https://github.com/apache/cassandra] repository. In this case Antora 
would run on the [cassandra/doc|https://github.com/apache/cassandra/doc] 
directory and pull in a UI bundle released on the GitHub 
[cassandra-website|https://github.com/apache/cassandra-website] repository. The 
content generated by Antora using the site.yml file located in this repo can be 
used to preview the docs for review.
 * All other non-versioned material, such as the blogs, development 
instructions, and third-party page would live in the GitHub 
[cassandra-website|https://github.com/apache/cassandra-website] repository. 
Antora will generate the full website using the site.yml located in this repo, 
using both as content sources the material located in both the 
[cassandra|https://github.com/apache/cassandra] and 
[cassandra-website|https://github.com/apache/cassandra-website] repositories.

*Design, content, and layout would remain the same*
 This proposal means the roles of each repository will change. In addition, the 
workflow to publish material will change as well. However, the website design, 
content, and layout will remain the same.

As an added bonus the tooling would allow for a live preview server that is 
fast and responsive. However, the time to build and generate the {{nodetool}} 
and Cassandra YAML AssciDoc files will still take in the order of minutes to 
complete.

*The [cassandra-website|https://github.com/apache/cassandra-website] repository 
would need to be modified*
 The following changes would need to be made to the 
[cassandra-website|https://github.com/apache/cassandra-website] repository:
||File/Directory||Action||Reason||
|[content/|https://github.com/apache/cassandra-website/content]|keep|Production 
site content served from this directory|
|[src/_data/|https://github.com/apache/cassandra-website/src/_data]|delete|_site.yml_
 and _antora.yml_ include this info|
|[src/_includes/|https://github.com/apache/cassandra-website/src/_includes]|delete|Replace
 with UI bundle components|
|[src/_layout/|https://github.com/apache/cassandra-website/src/_layout]|delete|Replace
 with 

[jira] [Commented] (CASSANDRA-16066) Update and rework cassandra-website material to work with Antora

[Lorina Poland (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186068#comment-17186068
 ] 

Lorina Poland commented on CASSANDRA-16066:
---

The plan is to have two slightly different Docker containers, one in 
cassandra/doc that can be used to build the docs for preview while doing 
documentation work, and one in cassandra-website that would build the docs 
along with the other files, with preview and production build options. So, the 
overlap is similar, but the site.yml on cassandra-website would include all 
content sources, whereas cassandra/doc would only include the docs content 
sources. Here's an example of part of the site.yml file:
{code:java}
content:
  sources:
  - url: https://github.com/polandll/cassandra.git
branches:
- 'doc_redo_asciidoc' // this is the equivalent of trunk (4.0)
- 'doc_redo_asciidoc3.11' // this is the equivalent of cassandra3.11
start_path: doc/source
  - url: https://github.com/polandll/cassandra-website.git
branches:
- test_new_doc_builder
start_path: source
{code}
The second content source listed here will only be included the site.yml file 
on cassandra-website. 

Please note that I've been doing testing, so I do have both in cassandra/doc at 
the moment on my working branch, but that will change. 

> Update and rework cassandra-website material to work with Antora
> 
>
> Key: CASSANDRA-16066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16066
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Anthony Grasso
>Priority: Normal
>
> *We want to modernise the way the website is built*
>  Rework the cassandra-website repository to generate a UI bundle containing 
> resources that Antora will use when generating the Cassandra documents and 
> website.
> *The existing method is starting to become dated*
>  The documentation and website templates are currently in markdown format. 
> Sphinx is used to generate the Cassandra documentation and Jekyll generates 
> the website content. One of the major issues with the existing website 
> tooling is that the live preview server (render site as it is being updated) 
> is really slow. There is a preview server that is really fast, however it is 
> unable to detect changes to the content and render automatically.
> *We are migrating the docs to be rendered with Antora*
>  The work in CASSANDRA-16029 is converting the document templates to AsciiDoc 
> format. Sphinx is being replaced by Antora to generate the documentation 
> content. This change has two advantages:
>  * More flexibility if the Apache Cassandra documentation look and feel needs 
> to be updated or redesigned.
>  * More modern look and feel to the documentation.
> *We can use Antora to generate the website as well*
>  Antora could also be used to generate the Cassandra website content. As 
> suggested on the [mailing 
> list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15577.html] 
> this would require the existing markdown templates to be converted to 
> AsciiDoc as well.
> *Antora needs a UI bundle to style content*
>  For Antora to generate the document content and potentially the website 
> content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the 
> HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and 
> (site-wide) images. As such, it provides both the visual theme and user 
> interactions for the documentation. Effectively the UI bundle is the 
> templates and styling that are applied to the documentation and website 
> content.
> *The [cassandra-website|https://github.com/apache/cassandra-website] 
> repository can be used to generate the UI bundle*
>  All the resources associated with templating and styling the documentation 
> and website can be placed in the 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> In this case the repository would serve two purposes;
>  * Generation of the UI bundle resources.
>  * Serve the production website content.
> *The [cassandra|https://github.com/apache/cassandra] repository would contain 
> the site material*
>  All other material that is used to generate documentation, blogs and project 
> information would live in the [cassandra|https://github.com/apache/cassandra] 
> repository. In this case Antora would run on the 
> [cassandra/doc|https://github.com/apache/cassandra/doc] directory and pull in 
> a UI bundle released on the GitHub 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> The content generated by Antora would be placed in the 
> [cassandra-website/content|https://github.com/apache/cassandra-website/content]
>  directory
> *Design, content, and layout would remain 

[jira] [Commented] (CASSANDRA-13701) Lower default num_tokens

[Adam Holmberg (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186065#comment-17186065
 ] 

Adam Holmberg commented on CASSANDRA-13701:
---

Linked the follow-on ticket for improving dtest runtime.

> Lower default num_tokens
> 
>
> Key: CASSANDRA-13701
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13701
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Chris Lohfink
>Assignee: Alexander Dejanovski
>Priority: Low
> Fix For: 4.0-alpha
>
>
> For reasons highlighted in CASSANDRA-7032, the high number of vnodes is not 
> necessary. It is very expensive for operations processes and scanning. Its 
> come up a lot and its pretty standard and known now to always reduce the 
> num_tokens within the community. We should just lower the defaults.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16080) Create docs for upgrading C*

[Lorina Poland (Jira)]

Lorina Poland created CASSANDRA-16080:
-

 Summary: Create docs for upgrading C*
 Key: CASSANDRA-16080
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16080
 Project: Cassandra
  Issue Type: New Feature
  Components: Documentation/Website
Reporter: Lorina Poland


Add documentation on upgrading Cassandra. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16079) Improve dtest runtime

[Adam Holmberg (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Holmberg updated CASSANDRA-16079:
--
Change Category: Quality Assurance
 Complexity: Normal
  Fix Version/s: 4.0-beta
 Status: Open  (was: Triage Needed)

> Improve dtest runtime
> -
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
> 30% increase in run time. While that change was accepted, we wanted to spin 
> out a ticket to optimize dtests in an attempt to gain back some of that 
> runtime.
> At this time we don't have concrete improvements in mind, so the first order 
> of this ticket will be to analyze the state of things currently, and try to 
> ascertain some valuable optimizations. Once the problems are understood, we 
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16079) Improve dtest runtime

[Adam Holmberg (Jira)]

Adam Holmberg created CASSANDRA-16079:
-

 Summary: Improve dtest runtime
 Key: CASSANDRA-16079
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
 Project: Cassandra
  Issue Type: Improvement
  Components: CI
Reporter: Adam Holmberg


A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a 
30% increase in run time. While that change was accepted, we wanted to spin out 
a ticket to optimize dtests in an attempt to gain back some of that runtime.

At this time we don't have concrete improvements in mind, so the first order of 
this ticket will be to analyze the state of things currently, and try to 
ascertain some valuable optimizations. Once the problems are understood, we 
will break down subtasks to divide the work.

Some areas to consider:
* cluster reuse
* C* startup optimizations
* Tests that should be ported to in-JVM dtest or even unit tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16066) Update and rework cassandra-website material to work with Antora

[Rahul Singh (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186063#comment-17186063
 ] 

Rahul Singh commented on CASSANDRA-16066:
-

Great! Looking forward to the POC. Where does the actual build of the docs 
happen. e.g. Will someone pull down cassandra-website and do edits/previews, or 
will they pull down cassandra .. I see that the dockerfile right now is in the 
main cassandra repo. 

> Update and rework cassandra-website material to work with Antora
> 
>
> Key: CASSANDRA-16066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16066
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Anthony Grasso
>Priority: Normal
>
> *We want to modernise the way the website is built*
>  Rework the cassandra-website repository to generate a UI bundle containing 
> resources that Antora will use when generating the Cassandra documents and 
> website.
> *The existing method is starting to become dated*
>  The documentation and website templates are currently in markdown format. 
> Sphinx is used to generate the Cassandra documentation and Jekyll generates 
> the website content. One of the major issues with the existing website 
> tooling is that the live preview server (render site as it is being updated) 
> is really slow. There is a preview server that is really fast, however it is 
> unable to detect changes to the content and render automatically.
> *We are migrating the docs to be rendered with Antora*
>  The work in CASSANDRA-16029 is converting the document templates to AsciiDoc 
> format. Sphinx is being replaced by Antora to generate the documentation 
> content. This change has two advantages:
>  * More flexibility if the Apache Cassandra documentation look and feel needs 
> to be updated or redesigned.
>  * More modern look and feel to the documentation.
> *We can use Antora to generate the website as well*
>  Antora could also be used to generate the Cassandra website content. As 
> suggested on the [mailing 
> list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15577.html] 
> this would require the existing markdown templates to be converted to 
> AsciiDoc as well.
> *Antora needs a UI bundle to style content*
>  For Antora to generate the document content and potentially the website 
> content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the 
> HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and 
> (site-wide) images. As such, it provides both the visual theme and user 
> interactions for the documentation. Effectively the UI bundle is the 
> templates and styling that are applied to the documentation and website 
> content.
> *The [cassandra-website|https://github.com/apache/cassandra-website] 
> repository can be used to generate the UI bundle*
>  All the resources associated with templating and styling the documentation 
> and website can be placed in the 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> In this case the repository would serve two purposes;
>  * Generation of the UI bundle resources.
>  * Serve the production website content.
> *The [cassandra|https://github.com/apache/cassandra] repository would contain 
> the site material*
>  All other material that is used to generate documentation, blogs and project 
> information would live in the [cassandra|https://github.com/apache/cassandra] 
> repository. In this case Antora would run on the 
> [cassandra/doc|https://github.com/apache/cassandra/doc] directory and pull in 
> a UI bundle released on the GitHub 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> The content generated by Antora would be placed in the 
> [cassandra-website/content|https://github.com/apache/cassandra-website/content]
>  directory
> *Design, content, and layout would remain the same*
>  This proposal means the roles of each repository will change. In addition, 
> the workflow to publish material will change as well. However, the website 
> design, content, and layout will remain the same.
> As an added bonus the tooling would allow for a live preview server that is 
> fast and responsive. However, the time to build and generate the {{nodetool}} 
> and Cassandra YAML AssciDoc files will still take in the order of minutes to 
> complete.
> *The [cassandra-website|https://github.com/apache/cassandra-website] 
> repository would need to be modified*
>  The following changes would need to be made to the 
> [cassandra-website|https://github.com/apache/cassandra-website] repository:
> ||File/Directory||Action||Reason||
> |[content/|https://github.com/apache/cassandra-website/content]|keep|Production
>  site content served from this directory|
> 

[jira] [Commented] (CASSANDRA-16036) Add flag to disable chunk cache and disable by default

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186046#comment-17186046
 ] 

David Capwell commented on CASSANDRA-16036:
---

[~jmeredithco] pushed changes to the yaml to document this flag.

> Add flag to disable chunk cache and disable by default
> --
>
> Key: CASSANDRA-16036
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16036
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: async-profile.collapsed.svg, 
> clustering-in-clause_latency_selects_baseline.png, 
> clustering-in-clause_latency_selects_baseline_attempt3.png, 
> clustering-in-clause_latency_under90_selects_baseline.png, 
> clustering-in-clause_latency_under90_selects_baseline_attempt3.png, 
> clustering-slice_latency_selects_baseline.png, 
> clustering-slice_latency_under90_selects_baseline.png, 
> medium-blobs_latency_selects_baseline.png, 
> medium-blobs_latency_under90_selects_baseline.png, 
> partition-single-row-read_latency_selects_baseline.png, 
> partition-single-row-read_latency_under90_selects_baseline.png
>
>
> Chunk cache is enabled by default and doesn’t have a flag to disable without 
> impacting networking.  In performance testing 4.0 against 3.0 I found that 
> reads were slower in 4.0 and after profiling found that the ChunkCache was 
> partially to blame; after disabling the chunk cache, read performance had 
> improved.
> {code}
> 40_w_cc-selects.hdr
> #[Mean= 11.50063, StdDeviation   = 13.44014]
> #[Max =482.41254, Total count=   316477]
> #[Buckets =   25, SubBuckets =   262144]
> 40_wo_cc-selects.hdr
> #[Mean=  9.82115, StdDeviation   = 10.14270]
> #[Max =522.36493, Total count=   317444]
> #[Buckets =   25, SubBuckets =   262144]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16036) Add flag to disable chunk cache and disable by default

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186039#comment-17186039
 ] 

David Capwell commented on CASSANDRA-16036:
---

Thanks jon, will update conf/cassandra.yaml.

bq. work out how to use their memory budget.

Yeah, this is why I still lean to this default.  I have not found a use case 
which benefits from chunk cache and find some use cases which are roughly equal 
with and without, so favor disabling in these cases to save memory (a much 
limited resource in production).

> Add flag to disable chunk cache and disable by default
> --
>
> Key: CASSANDRA-16036
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16036
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: async-profile.collapsed.svg, 
> clustering-in-clause_latency_selects_baseline.png, 
> clustering-in-clause_latency_selects_baseline_attempt3.png, 
> clustering-in-clause_latency_under90_selects_baseline.png, 
> clustering-in-clause_latency_under90_selects_baseline_attempt3.png, 
> clustering-slice_latency_selects_baseline.png, 
> clustering-slice_latency_under90_selects_baseline.png, 
> medium-blobs_latency_selects_baseline.png, 
> medium-blobs_latency_under90_selects_baseline.png, 
> partition-single-row-read_latency_selects_baseline.png, 
> partition-single-row-read_latency_under90_selects_baseline.png
>
>
> Chunk cache is enabled by default and doesn’t have a flag to disable without 
> impacting networking.  In performance testing 4.0 against 3.0 I found that 
> reads were slower in 4.0 and after profiling found that the ChunkCache was 
> partially to blame; after disabling the chunk cache, read performance had 
> improved.
> {code}
> 40_w_cc-selects.hdr
> #[Mean= 11.50063, StdDeviation   = 13.44014]
> #[Max =482.41254, Total count=   316477]
> #[Buckets =   25, SubBuckets =   262144]
> 40_wo_cc-selects.hdr
> #[Mean=  9.82115, StdDeviation   = 10.14270]
> #[Max =522.36493, Total count=   317444]
> #[Buckets =   25, SubBuckets =   262144]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16078) Performance regression for queries accessing multiple rows

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16078:
--
 Bug Category: Parent values: Degradation(12984)Level 1 values: Performance 
Bug/Regression(12997)
   Complexity: Normal
Discovered By: Performance Regression Test
Fix Version/s: 4.0-beta
 Severity: Critical
   Status: Open  (was: Triage Needed)

> Performance regression for queries accessing multiple rows
> --
>
> Key: CASSANDRA-16078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16078
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> This is spin off from CASSANDRA-16036.
> In testing 4.0 relative to 3.0* I found that queries which accessed multiple 
> rows to have a noticeable performance decrease; two queries were used in the 
> test (more may be impacted, others might not): query partition (table has 
> clustering keys) with LIMIT, and query clustering keys using IN clause.
> In the below graphs the green line is 3.0 and the other lines are 4.0 (with 
> and without chunk cache)
> Partition with LIMIT
> !https://issues.apache.org/jira/secure/attachment/13009751/clustering-slice_latency_selects_baseline.png!
> !https://issues.apache.org/jira/secure/attachment/13009750/clustering-slice_latency_under90_selects_baseline.png!
> Cluster with IN clause
> !https://issues.apache.org/jira/secure/attachment/13009749/clustering-in-clause_latency_selects_baseline.png!
> !https://issues.apache.org/jira/secure/attachment/13009748/clustering-in-clause_latency_under90_selects_baseline.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16078) Performance regression for queries accessing multiple rows

[David Capwell (Jira)]

David Capwell created CASSANDRA-16078:
-

 Summary: Performance regression for queries accessing multiple rows
 Key: CASSANDRA-16078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16078
 Project: Cassandra
  Issue Type: Bug
  Components: Consistency/Coordination
Reporter: David Capwell


This is spin off from CASSANDRA-16036.

In testing 4.0 relative to 3.0* I found that queries which accessed multiple 
rows to have a noticeable performance decrease; two queries were used in the 
test (more may be impacted, others might not): query partition (table has 
clustering keys) with LIMIT, and query clustering keys using IN clause.

In the below graphs the green line is 3.0 and the other lines are 4.0 (with and 
without chunk cache)


Partition with LIMIT

!https://issues.apache.org/jira/secure/attachment/13009751/clustering-slice_latency_selects_baseline.png!
!https://issues.apache.org/jira/secure/attachment/13009750/clustering-slice_latency_under90_selects_baseline.png!

Cluster with IN clause

!https://issues.apache.org/jira/secure/attachment/13009749/clustering-in-clause_latency_selects_baseline.png!
!https://issues.apache.org/jira/secure/attachment/13009748/clustering-in-clause_latency_under90_selects_baseline.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16066) Update and rework cassandra-website material to work with Antora

[Lorina Poland (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186032#comment-17186032
 ] 

Lorina Poland commented on CASSANDRA-16066:
---

Antora can build different styling "templates" for various pages. It can also 
use multiple repos for the content to be built. For instance, the non-doc pages 
can be stored in the cassandra-website repo and styled as, for instance, blog 
posts or non-doc pages. The docs, meanwhile, live in the cassandra/doc repo and 
are built with doc styling. 

Antora is quite versatile for the UI.

I've already started converting the cassandra-website pages to adoc (md->adoc 
is simple), and added the blog posts and code development information to that, 
along with the Community, Download, 3rd party, Contact Us, C* plug-ins, 
glossary, and native protocol specs (v.3,4,5).

> Update and rework cassandra-website material to work with Antora
> 
>
> Key: CASSANDRA-16066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16066
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Anthony Grasso
>Priority: Normal
>
> *We want to modernise the way the website is built*
>  Rework the cassandra-website repository to generate a UI bundle containing 
> resources that Antora will use when generating the Cassandra documents and 
> website.
> *The existing method is starting to become dated*
>  The documentation and website templates are currently in markdown format. 
> Sphinx is used to generate the Cassandra documentation and Jekyll generates 
> the website content. One of the major issues with the existing website 
> tooling is that the live preview server (render site as it is being updated) 
> is really slow. There is a preview server that is really fast, however it is 
> unable to detect changes to the content and render automatically.
> *We are migrating the docs to be rendered with Antora*
>  The work in CASSANDRA-16029 is converting the document templates to AsciiDoc 
> format. Sphinx is being replaced by Antora to generate the documentation 
> content. This change has two advantages:
>  * More flexibility if the Apache Cassandra documentation look and feel needs 
> to be updated or redesigned.
>  * More modern look and feel to the documentation.
> *We can use Antora to generate the website as well*
>  Antora could also be used to generate the Cassandra website content. As 
> suggested on the [mailing 
> list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15577.html] 
> this would require the existing markdown templates to be converted to 
> AsciiDoc as well.
> *Antora needs a UI bundle to style content*
>  For Antora to generate the document content and potentially the website 
> content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the 
> HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and 
> (site-wide) images. As such, it provides both the visual theme and user 
> interactions for the documentation. Effectively the UI bundle is the 
> templates and styling that are applied to the documentation and website 
> content.
> *The [cassandra-website|https://github.com/apache/cassandra-website] 
> repository can be used to generate the UI bundle*
>  All the resources associated with templating and styling the documentation 
> and website can be placed in the 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> In this case the repository would serve two purposes;
>  * Generation of the UI bundle resources.
>  * Serve the production website content.
> *The [cassandra|https://github.com/apache/cassandra] repository would contain 
> the site material*
>  All other material that is used to generate documentation, blogs and project 
> information would live in the [cassandra|https://github.com/apache/cassandra] 
> repository. In this case Antora would run on the 
> [cassandra/doc|https://github.com/apache/cassandra/doc] directory and pull in 
> a UI bundle released on the GitHub 
> [cassandra-website|https://github.com/apache/cassandra-website] repository. 
> The content generated by Antora would be placed in the 
> [cassandra-website/content|https://github.com/apache/cassandra-website/content]
>  directory
> *Design, content, and layout would remain the same*
>  This proposal means the roles of each repository will change. In addition, 
> the workflow to publish material will change as well. However, the website 
> design, content, and layout will remain the same.
> As an added bonus the tooling would allow for a live preview server that is 
> fast and responsive. However, the time to build and generate the {{nodetool}} 
> and Cassandra YAML AssciDoc files will still take in the order of minutes to 
> complete.
> *The 

[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells

[Blake Eggleston (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186021#comment-17186021
 ] 

Blake Eggleston commented on CASSANDRA-15393:
-

That's reasonable, I'll ping the dev list.

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186017#comment-17186017
 ] 

Michael Semb Wever commented on CASSANDRA-15393:


[~bdeggleston], a patch with 208 files and ~3.5k LOC, [~snazy] expressing 
concern, and not seeing any comment of why this was placed in 4.0-beta in May. 
That should be enough that our normal community response is to double check 
with a broader base. Maybe [~JoshuaMcKenzie] you can provide support and the 
reasoning why this was, and should be, in 4.0-beta?

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells

[Blake Eggleston (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186013#comment-17186013
 ] 

Blake Eggleston commented on CASSANDRA-15393:
-

[~mck], this ticket was been triaged to 4.0-beta back in May. Performance 
improvements aren't restricted by the feature freeze. Do you have concerns 
about this ticket that aren't related to the feature freeze?

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15393) Add byte array backed cells

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185997#comment-17185997
 ] 

Michael Semb Wever edited comment on CASSANDRA-15393 at 8/27/20, 5:19 PM:
--

{quote}Is there's a big difference between allowing performance improvements, 
especially ones that address real operational problems, into {{beta}} vs. 
{{4.0.x?}}
{quote}
The choice isn't binary, 4.1 is more appropriate (IMHO). Was just my way of 
stating my concerns about this going into 4.0-beta when so much effort has been 
put into the feature freeze. 4.0.x would still require a waiver IMO, but with 
the QA/testing lifecycle guidelines from 4.0-beta better landed, and the stress 
and anxiety of getting 4.0 out after so long, it may then be a much easier 
discussion to have.

Again, can we please take this to the ML, and leave the ticket for the 
technical?


was (Author: michaelsembwever):
{quote}Is there's a big difference between allowing performance improvements, 
especially ones that address real operational problems, into {{beta}} vs. 
{{4.0.x?}}
{quote}
The choice isn't binary, 4.1 is more appropriate. Was just my way of stating my 
concerns about this going into 4.0-beta when so much effort has been put into 
the feature freeze. 4.0.x would still require a waiver IMO, but with the 
QA/testing lifecycle guidelines from 4.0-beta better landed, and the stress and 
anxiety of getting 4.0 out after so long, it may then be a much easier 
discussion to have.

Again, can we please take this to the ML, and leave the ticket for the 
technical?

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185997#comment-17185997
 ] 

Michael Semb Wever commented on CASSANDRA-15393:


{quote}Is there's a big difference between allowing performance improvements, 
especially ones that address real operational problems, into {{beta}} vs. 
{{4.0.x?}}
{quote}
The choice isn't binary, 4.1 is more appropriate. Was just my way of stating my 
concerns about this going into 4.0-beta when so much effort has been put into 
the feature freeze. 4.0.x would still require a waiver IMO, but with the 
QA/testing lifecycle guidelines from 4.0-beta better landed, and the stress and 
anxiety of getting 4.0 out after so long, it may then be a much easier 
discussion to have.

Again, can we please take this to the ML, and leave the ticket for the 
technical?

> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers 
> have a fairly high overhead given how frequently they’re used, and on the 
> compaction and local read path we don’t do anything that needs them. Use of 
> byte buffer methods only happens on the coordinator. Using cells that are 
> backed by byte arrays instead in these situations reduces compaction and read 
> garbage up to 22% in many cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15950) unable to start the cassandra on windows 10 machine

[Scott.Wu (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185977#comment-17185977
 ] 

Scott.Wu commented on CASSANDRA-15950:
--

Same issue appeared in 3.11.7 (I am using win 10 with JDK 1.8.0_361)

 

INFO [main] 2020-08-27 13:00:49,647 SigarLibrary.java:44 - Initializing SIGAR 
library
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x10014ed4, pid=39744, 
tid=0x9794
#
# JRE version: Java(TM) SE Runtime Environment (8.0_261-b12) (build 
1.8.0_261-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode 
windows-amd64 compressed oops)
# Problematic frame:
# C [sigar-amd64-winnt.dll+0x14ed4]
#
# Failed to write core dump. Minidumps are not enabled by default on client 
versions of Windows
#
# An error report file with more information is saved as:
# C:\DevEnv\apache-cassandra-3.11.7\bin\hs_err_pid39744.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

> unable to start the cassandra on windows 10 machine
> ---
>
> Key: CASSANDRA-15950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15950
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mandar Kulkarni
>Priority: Normal
>
> Os - Windows 10
> Cassandra -  apache-cassandra-3.11.4
> java version - java version "1.8.0_261"
> Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)
> python version - Python 3.8.4
> I am getting below error when starting cassandra -f
> INFO [main] 2020-07-15 17:20:54,566 SigarLibrary.java:44 - Initializing SIGAR 
> library
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x10014ed4, 
> pid=13704, tid=0x25f4
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_261-b12) (build 
> 1.8.0_261-b12)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode 
> windows-amd64 compressed oops)
> # Problematic frame:
> # C [sigar-amd64-winnt.dll+0x14ed4]
> #
> # Failed to write core dump. Minidumps are not enabled by default on client 
> versions of Windows
> #
> # An error report file with more information is saved as:
> # C:\cass311\apache-cassandra-3.11.4\bin\hs_err_pid13704.log
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16077) Only allow strings to be passed to JMX authentication

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16077:
-
Change Category: Operability
 Complexity: Low Hanging Fruit
  Fix Version/s: 2.1.22
 2.2.18
 4.0-beta2
 3.11.8
 Status: Open  (was: Triage Needed)

> Only allow strings to be passed to JMX authentication
> -
>
> Key: CASSANDRA-16077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16077
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/JMX
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.8, 4.0-beta2, 2.2.18, 2.1.22
>
>
> It doesn't make sense to allow other object types.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16077) Only allow strings to be passed to JMX authentication

[Brandon Williams (Jira)]

Brandon Williams created CASSANDRA-16077:


 Summary: Only allow strings to be passed to JMX authentication
 Key: CASSANDRA-16077
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16077
 Project: Cassandra
  Issue Type: Improvement
  Components: Observability/JMX
Reporter: Brandon Williams
Assignee: Brandon Williams


It doesn't make sense to allow other object types.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15406) Show the progress of data streaming and index build

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185921#comment-17185921
 ] 

Benjamin Lerer commented on CASSANDRA-15406:


{quote}However there is a weird clash when this is computed because Netstats 
reports the total bytes to be sent as total over this "sections size" even it 
is sent over like compressed, it is compressed by default by 
CassandraCompressedStreamWriter and the individual "total bytes" per each item 
to be streamed is taken from its totalSize method and there it is computed as 
compressed so the numbers dont match.{quote}

In {{CassandraStreamHeader}} if no {{CompressionInfo}} has been provided via 
the {{CassandraStreamHeader.Builder}} the {{size}} field will be computed in 
the constructor assuming a {{null}} {{CompressionInfo}}. As 
{{CassandraOutgoingFile}} creates a {{CassandraStreamHeader}} without providing 
a {{CompressionInfo}} the size computed will always be wrong for compressed 
streams.

The correct way to fix the code is in my opinion to remove from the constructor 
and the builder the {{compressionInfo}} parameter, to remove the 
{{calculateCompressionInfo()}} method and to compute the {{compressionInfo}} 
field value within the constructor.

The {{transient}} keyword from the {{compressionInfo}} field can also be 
removed as the class does not implement {{Serializable}}.

I also noticed that there is an inconsitency between the 
{{CassandraOutgoingFile#write}} and the {{CassandraStreamHeader#calculateSize}} 
methods.
If the {{shouldStreamEntireSSTable}} is {{true}} but the {{out}} parameter from 
the {{write}} method is not an {{AsyncStreamingOutputPlus}} instance the 
computed size will be the wrong one. I did not investigate in which case this 
scenario can occurs but we should probably check it too.

> Show the progress of data streaming and index build 
> 
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming, Legacy/Streaming and Messaging, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> I found that we should supply a command to show the progress of streaming 
> when we do the operation of bootstrap/move/decommission/removenode. For when 
> do data streaming , noboday knows which steps there program are in , so I 
> think a command to show the joing/leaving node's is needed .
>  
> PR [https://github.com/apache/cassandra/pull/558]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14922) In JVM dtests need to clean up after instance shutdown

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-14922:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> In JVM dtests need to clean up after instance shutdown
> --
>
> Key: CASSANDRA-14922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14922
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Low
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: AllThreadsStopped.png, ClassLoadersRetaining.png, 
> LeakedNativeMemory.png, Leaking_Metrics_On_Shutdown.png, 
> MainClassRetaining.png, MemoryReclaimedFix.png, 
> Metaspace_Actually_Collected.png, OnlyThreeRootsLeft.png, Screen Shot 
> 2019-01-30 at 15.46.35.png, Screen Shot 2019-01-30 at 15.47.13.png, 
> no_more_references.png
>
>
> Currently the unit tests are failing on circleci ([example 
> one|https://circleci.com/gh/jolynch/cassandra/300#tests/containers/1], 
> [example 
> two|https://circleci.com/gh/rustyrazorblade/cassandra/44#tests/containers/1]) 
> because we use a small container (medium) for unit tests by default and the 
> in JVM dtests are leaking a few hundred megabytes of memory per test right 
> now. This is not a big deal because the dtest runs with the larger containers 
> continue to function fine as well as local testing as the number of in JVM 
> dtests is not yet high enough to cause a problem with more than 2GB of 
> available heap. However we should fix the memory leak so that going forwards 
> we can add more in JVM dtests without worry.
> I've been working with [~ifesdjeen] to debug, and the issue appears to be 
> unreleased Table/Keyspace metrics (screenshot showing the leak attached). I 
> believe that we have a few potential issues that are leading to the leaks:
> 1. The 
> [{{Instance::shutdown}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/Instance.java#L328-L354]
>  method is not successfully cleaning up all the metrics created by the 
> {{CassandraMetricsRegistry}}
>  2. The 
> [{{TestCluster::close}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/TestCluster.java#L283]
>  method is not waiting for all the instances to finish shutting down and 
> cleaning up before continuing on
> 3. I'm not sure if this is an issue assuming we clear all metrics, but 
> [{{TableMetrics::release}}|https://github.com/apache/cassandra/blob/4ae229f5cd270c2b43475b3f752a7b228de260ea/src/java/org/apache/cassandra/metrics/TableMetrics.java#L951]
>  does not release all the metric references (which could leak them)
> I am working on a patch which shuts down everything and assures that we do 
> not leak memory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14821) Make it possible to run multi-node coordinator/replica tests in a single JVM

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-14821:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Make it possible to run multi-node coordinator/replica tests in a single JVM
> 
>
> Key: CASSANDRA-14821
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14821
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 4.0, 4.0-alpha1
>
>
> This patch proposes an in-JVM Distributed Tester that can help to write 
> distributed tests in a single JVM and be able to control node behaviour in a 
> fine-grained way and set up nodes exactly how one needs it: configuration 
> settings, parameters, which are also controllable in runtime on a per node 
> basis, so each node can have its own unique state.
> It fires up multiple Cassandra Instances in a single JVM. It is done through 
> having distinct class loaders in order to work around the singleton problem 
> in Cassandra. In order to be able to pass some information between the nodes, 
> a common class loader is used that loads up java standard library and several 
> helper classes. Tests look a lot like CQLTester tests would usually look like.
> Each Cassandra Instance, with its distinct class loader is using 
> serialisation and class loading mechanisms in order to run instance-local 
> queries and execute node state manipulation code, hooks, callbacks etc.
> First version mocks out Messaging Service and simplifies schema management by 
> simply running schema change commands on each of the instances separately. 
> Internode communication is mocked by passing ByteBuffers through shared class 
> loader.
> |[patch|https://github.com/ifesdjeen/cassandra/tree/14821]|[tests|https://circleci.com/workflow-run/b371e72d-9dc8-455d-acb3-45cd8be1386b]|



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14931) Backport In-JVM dtests to 2.2, 3.0 and 3.11

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-14931:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Backport In-JVM dtests to 2.2, 3.0 and 3.11
> ---
>
> Key: CASSANDRA-14931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14931
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 2.2.14, 3.0.18, 3.11.4
>
>
> The In-JVM dtests are of significant value, and much of the testing we are 
> exploring with it can easily be utilised on all presently maintained 
> versions.  We should backport the functionality to at least 3.0.x and 3.11.x 
> - and perhaps even consider 2.2.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14937) Multi-version In-JVM dtests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-14937:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Multi-version In-JVM dtests
> ---
>
> Key: CASSANDRA-14937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14937
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Test/dtest/java
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0, 4.0-alpha1
>
>
> In order to support more sophisticated upgrade tests, including complex fuzz 
> tests that can span a sequence of version upgrades, we propose abstracting a 
> cross-version API for the in-jvm dtests.  This will permit starting a node 
> with an arbitrary compatible C* version, stopping the node, and restarting it 
> with another C* version.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14969) Clean up ThreadLocals directly instead of via reflection

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-14969:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Clean up ThreadLocals directly instead of via reflection
> 
>
> Key: CASSANDRA-14969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Joey Lynch
>Priority: Low
>
> In CASSANDRA-14922 we have to institute a bit of a hack via reflection to 
> clean up thread local variables that are not properly {{destroyed}} in 
> {{DistributedTestBase::cleanup}}. Let's make sure that all of the thread 
> locals we have are cleaned up via {{destroy}} calls instead of relying on 
> reflection here.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14974) Unit test stdout capture is malfunctioning in 4.0

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-14974:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Unit test stdout capture is malfunctioning in 4.0
> -
>
> Key: CASSANDRA-14974
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14974
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java, Test/unit
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 4.0-alpha1
>
>
> In 3.x unit tests we make sure to capture stdout to the unit test files, in 
> case tests log to stdout and not to a logger.  However, in 4.0 due to a 
> configuration parameter that is deprecated in logback, the logic is 
> short-circuited silently.
> Once fixed, this affects the cleanup of in-jvm dtests which would register an 
> infinite chain of System.out overrides (one for each node), so we make the 
> functionality robust to multiple instantiations, as well as improve its 
> startup/shutdown sequence guarantees.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15062) Use the correct readCommand when making local read requests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15062:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Use the correct readCommand when making local read requests
> ---
>
> Key: CASSANDRA-15062
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15062
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> We are currently not using the provided {{readCommand}} when doing local 
> requests in {{AbstractReadExecutor.makeRequests}} - in trunk this makes us 
> lose the {{acceptsTransient}} flag and in other versions it makes us do data 
> reads instead of digest ones (which is not a problem but a bit confusing 
> considering the log message)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15148) Add in-jvm dtest for Gossip to trunk

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15148:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Add in-jvm dtest for Gossip to trunk
> 
>
> Key: CASSANDRA-15148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15148
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java
>Reporter: Sam Tunnicliffe
>Priority: Normal
>
> When merging CASSANDRA-15120 to trunk, the in-jvm dtest support for gossip 
> and networking were ommitted as much of the work to support that is already 
> done and due to land in CASSANDRA-15066. Once that patch is committed, we 
> should add the {{GossipTest}} from CASSANDRA-15120 to trunk.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15078:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Support cross version messaging in in-jvm upgrade dtests
> 
>
> Key: CASSANDRA-15078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15078
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0, 4.0-alpha1
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15170:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Reduce the time needed to release in-JVM dtest cluster resources after close
> 
>
> Key: CASSANDRA-15170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15170
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0, 4.0-alpha1
>
>
> There are a few issues that slow the in-JVM dtests from reclaiming metaspace 
> once the cluster is closed.
> IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes 
> this thread was still running 10s after the dtest cluster was closed.  
> Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that 
> the thread executing the class loader close executes sooner.
> If an OutboundTcpConnection is waiting to connect() and the endpoint is not 
> answering, it has to wait for a timeout before it exits. Instead it should 
> check the isShutdown flag and terminate early if shutdown has been requested.
> In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a 
> try-with-resources construct and leaks a file handle for the directory.  This 
> doesn't matter for normal usage, it leaks a file handle for each dtest 
> Instance created.
> On trunk, Netty global event executor threads are still running and delay GC 
> for the instance class loader.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15147) Backport in-jvm dtest feature flags to 2.2

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15147:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Backport in-jvm dtest feature flags to 2.2
> --
>
> Key: CASSANDRA-15147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15147
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java
>Reporter: Sam Tunnicliffe
>Priority: Normal
>
> CASSANDRA-15120 adds gossip and networking to the in-jvm dtests and provides 
> feature flags to enable them in tests. This includes changes to the 
> {{Cluster}} interface which need to be backported to 2.2 to support cross 
> version tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15239) [flaky in-mem dtest] nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15239:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> [flaky in-mem dtest] nodeDownDuringMove - 
> org.apache.cassandra.distributed.test.GossipTest
> --
>
> Key: CASSANDRA-15239
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15239
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Jay Zhuang
>Priority: Normal
>
> The in-mem dtest fail from time to time:
> {noformat}
> nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to 
> contact any seeds!
> {noformat}
> [https://circleci.com/gh/Instagram/cassandra/98]
> More details:
> {noformat}
> Testcase: 
> nodeDownDuringMove(org.apache.cassandra.distributed.test.GossipTest): Caused 
> an ERROR
> java.lang.IllegalStateException: Unable to contact any seeds!
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to 
> contact any seeds!
> at 
> org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:166)
> at 
> org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$4(IsolatedExecutor.java:69)
> at 
> org.apache.cassandra.distributed.impl.Instance.startup(Instance.java:322)
> at 
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:148)
> at 
> org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove(GossipTest.java:96)
> Caused by: java.lang.IllegalStateException: Unable to contact any seeds!
> at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1261)
> at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:921)
> at 
> org.apache.cassandra.distributed.impl.Instance.lambda$startup$6(Instance.java:301)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
> at java.lang.Thread.run(Thread.java:748)
> Test org.apache.cassandra.distributed.test.GossipTest FAILED
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15240) Reinstate support for native libraries for in-JVM dtests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15240:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Reinstate support for native libraries for in-JVM dtests
> 
>
> Key: CASSANDRA-15240
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15240
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Jon Meredith
>Priority: Normal
>
> While working on CASSANDRA-15170 native libraries for libc functions, epoll 
> support and openssl were observed holding gcroots to the instance class 
> loaders when in-JVM dtest {{with(NETWORK)}} support was enabled. The solution 
> for CASSANDRA-15170 was to disable native libraries to get everything 
> working, but this is not ideal because in-JVM tests will not be testing the 
> real code on that platform.
> One proposed solution from [~ifesdjeen] and [~benedict] is to introduce an 
> additional classloader per-Cassandra version that can be used for loading 
> native libraries and share the {{CassandraVersionClassLoader}} by each 
> instance of that version, enabling the {{InstanceClassLoader}} to be garbage 
> collected.
> {noformat}
> CLibrary 
>  com.sun.jna.Native.registeredClasses
>  com.sun.jna.Native.options
>  com.sun.jna.Native.registredLibraries
> Netty
>io.netty.channel.ChannelException
>io.netty.channel.unix.DatagramSocketAddress
>io.netty.channel.unix.PeerCredentials
>io.netty.internal.tcnative.CertificateCallbackTask
>io.netty.internal.tcnative.CertificateVerifierTask
>io.netty.internal.tcnative.SSLPrivateKeyMethodDecryptTask
>io.netty.internal.tcnative.SSLPrivateKeyMethodSignTask
>io.netty.internal.tcnative.SSLPrivateKeyMethodTask
>io.netty.internal.tcnative.SSLTask
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15319) Add support for network topology and tracing to in-JVM dtests.

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15319:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Add support for network topology and tracing to in-JVM dtests.
> --
>
> Key: CASSANDRA-15319
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15319
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> While working on CASSANDRA-15318, testing it properly with an in-JVM test 
> requires setting up the network topology and tracing requests to check which 
> nodes performed forwarding.
>   
> In support of testing, make it possible to create in-JVM clusters with nodes 
> appearing in different datacenter/racks and add support for executing queries 
> with tracing enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15171) In-JVM dtests leak file handles after closing cluster

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15171:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

>  In-JVM dtests leak file handles after closing cluster
> --
>
> Key: CASSANDRA-15171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15171
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>
> From the work in CASSANDRA-15170, reviewing the output from lsof shows that 
> file descriptors from closed clusters are still open after all instance 
> objects and the instance class loader have been garbage collected.
> To run sustained fuzz/property based testing, file handle leaks need to be 
> minimized.
> Reproduce by running ResourceLeakTest.looperTest included as part of 
> CASSANDRA-15170 and look at `build/test/*-lsof-final.txt`



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15313:

Component/s: (was: Test/dtest/python)
 Test/unit

> Fix flaky - ChecksummingTransformerTest - 
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure: 
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression - 
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>   at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
>   at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
>   at org.quicktheories.impl.Precursor.(Precursor.java:17)
>   at 
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
>   at 
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
>   at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
>   at 
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
>   at 
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown 
> Source)
>   at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
>   at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
>   at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown 
> Source)
>   at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
>   at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown 
> Source)
>   at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
>   at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown 
> Source)
>   at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
>   at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown 
> Source)
>   at 
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
>   at 
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown 
> Source)
>   at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
>   at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
>   at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
>   at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown 
> Source)
>   at org.quicktheories.impl.Core.generate(Core.java:150)
>   at org.quicktheories.impl.Core.shrink(Core.java:103)
>   at org.quicktheories.impl.Core.run(Core.java:39)
>   at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
>   at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
>   at 
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
>   at 
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15371) Incorrect messaging service version breaks in-JVM upgrade tests on trunk

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15371:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Incorrect messaging service version breaks in-JVM upgrade tests on trunk
> 
>
> Key: CASSANDRA-15371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15371
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha3
>
>
> The in-JVM upgrade tests on trunk currently fail because the messaging
>  version for internode messaging is selected as 
> {{MessagingService.current_version}},
>  a regression from the implementation in CASSANDRA-15078.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15347) Add client testing capabilities to in-jvm tests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15347:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Add client testing capabilities to in-jvm tests
> ---
>
> Key: CASSANDRA-15347
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15347
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Doug Rohrer
>Priority: Normal
>  Labels: patch-available, pull-request-available
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha3
>
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> Allow testing native transport code path using in-jvm tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15329) in-JVM dtest fails on java 11 since system ClassLoader is not a URLClassLoader

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15329:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> in-JVM dtest fails on java 11 since system ClassLoader is not a URLClassLoader
> --
>
> Key: CASSANDRA-15329
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15329
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Attachments: CASSANDRA-15329.patch
>
>
> When running the in-JVM dtests on on java 11 they fail while trying to cast 
> the Versions.class.getClassLoader to URLClassLoader, which is no longer the 
> default ClassLoader on java 11.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15386) Use multiple data directories in the in-jvm dtests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15386:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Use multiple data directories in the in-jvm dtests
> --
>
> Key: CASSANDRA-15386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15386
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Compaction, Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.11.x, 4.x
>
>
> We should default to using 3 data directories when running the in-jvm dtests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15383) Unable to init Verb in jvm-dtest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15383:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Unable to init Verb in jvm-dtest
> 
>
> Key: CASSANDRA-15383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15383
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 4.0-alpha3
>
>
> Verb cannot be initialized in dtest. The test cases that has references to 
> Verb get the following error
> {code:java}
> java.lang.ExceptionInInitializerError
>   at org.apache.cassandra.net.Verb.(Verb.java:87)
>   at 
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.failingReadRepairTest(DistributedReadWritePathTest.java:134)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.cassandra.config.DatabaseDescriptor.getConcurrentReaders(DatabaseDescriptor.java:1611)
>   at org.apache.cassandra.concurrent.Stage.(Stage.java:48)
> {code}
> It is caused by the DatabaseDescriptor being uninitialized and NPE was 
> thrown. The dtest initializes the DatabaseDescriptor at instance-level with 
> the isolated InstanceClassLoader. Since test code is not loaded by the 
> InstanceClassLoader, it does not know the already initialized 
> DatabaseDescriptor.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15450) in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the wrong instance, it uses cluster generation when it should be using the instance id

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15450:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the 
> wrong instance, it uses cluster generation when it should be using the 
> instance id
> ---
>
> Key: CASSANDRA-15450
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15450
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 4.0-alpha3
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> In AbstractCluster.uncaughtExceptions, we attempt to get the instance from 
> the class loader and used the “generation”. This value is actually the 
> cluster id, so causes tests to fail when multiple tests share the same JVM; 
> it should be using the “id” field which represents the instance id relative 
> to the cluster.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15460) Fix missing call to enable RPC after native transport is started in in-jvm dtests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15460:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Fix missing call to enable RPC after native transport is started in in-jvm 
> dtests
> -
>
> Key: CASSANDRA-15460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15460
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Doug Rohrer
>Assignee: Doug Rohrer
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha3
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When starting the native transport, the original patch missed the step of 
> calling {{StorageService.instance.setRpcReady(true);}}. This appears to only 
> be required for counter columns, but without it you can't update a counter 
> value.
> We should add this call after starting up the native transport, and set it to 
> {{false}} during the shutdown sequence to mimic the production code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15447) in-jvm dtest support for subnets doesn't change seed provider subnet

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15447:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> in-jvm dtest support for subnets doesn't change seed provider subnet
> 
>
> Key: CASSANDRA-15447
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15447
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Doug Rohrer
>Assignee: Doug Rohrer
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x, 4.0-alpha3
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> When using the `withSubnet` function on AbstractCluster.Builder, the 
> newly-selected subnet is never used when setting up the SeedProvider in the 
> constructor of InstanceConfig, which is hard-coded to 127.0.0.1. Because of 
> this, clusters with any subnet other than 0, and gossip enabled, cannot start 
> up as they have no seed provider in their subnet and what should be the seed 
> (instance 1) doesn't think it is the seed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15434) Make test module for nodetools

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15434:

Component/s: Test/dtest/java

> Make test module for nodetools 
> ---
>
> Key: CASSANDRA-15434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15434
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java, Test/dtest/python, Test/unit, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: maxwellguo
>Priority: Low
>
> All nodetools feature don't got a unit/dtest module. We should add one for it



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15456) Backport CASSANDRA-15332 dtest changes to 3.x

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15456:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Backport CASSANDRA-15332 dtest changes to 3.x
> -
>
> Key: CASSANDRA-15456
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15456
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>
> When CASSANDRA-15332 was merged there was no work to back port the dtest 
> changes to 3.x.  As of this moment, we want to make sure 3.x and 4.x dtest 
> share the same API, so since CASSANDRA-15332 changed the API we need to make 
> sure those changes are reflected in 3.x



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15429) Support NodeTool for in-jvm dtest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15429:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Support NodeTool for in-jvm dtest
> -
>
> Key: CASSANDRA-15429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15429
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Test/dtest/java
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 4.0-alpha3
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> In-JVM dtest framework does not support nodetool as of now. This 
> functionality is wanted in some tests, e.g. constructing an end-to-end test 
> scenario that uses nodetool.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15502) In Tree Tooling with Java 11

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15502:

Component/s: Test/dtest/java

> In Tree Tooling with Java 11
> 
>
> Key: CASSANDRA-15502
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15502
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python, Test/fuzz, Tool/bulk 
> load, Tool/cqlsh, Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, 
> Tool/stress
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> This is to cover testing the various tools running on java 11.
> The scope of this testing is manual testing and not automated, different 
> JIRAs should cover automation testing these tool.
> The tools in question are: nodetool, sstableloader, sstablescrub, 
> sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, 
> compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, 
> sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these 
> may be tested already in dtest)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15497) Implement node bootstrap in in-JVM tests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15497:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Implement node bootstrap in in-JVM tests
> 
>
> Key: CASSANDRA-15497
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15497
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: High
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha4
>
>
> Currently, we do not have an ability to add nodes to the running in-jvm 
> cluster, either by bootstrap or replacement process. We need to add an 
> ability to add nodes in inactive state, start them up, and bootstrap to test 
> streaming, range movements, and operations that occur during these processes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15490) in-jvm dtest upgrade tests don't support 4.0 versioning

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15490:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> in-jvm dtest upgrade tests don't support 4.0 versioning
> ---
>
> Key: CASSANDRA-15490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15490
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Priority: Normal
>
> The current 4.0 versioning has a tag to denote that 4.0 isn’t in public 
> release yet (currently beta3).  dtest does version parsing which is much more 
> restive and doesn’t support preRelease tags (like the client version parsing 
> does), so skips the 4.0 jar.  This then leads to a exception while running 
> the upgrade tests
>  
> {code}
> java.lang.RuntimeException: No v4 versions found
>   at 
> org.apache.cassandra.distributed.impl.Versions.lambda$getLatest$2(Versions.java:166)
>   at java.util.Optional.orElseThrow(Optional.java:290)
>   at 
> org.apache.cassandra.distributed.impl.Versions.getLatest(Versions.java:166)
>   at 
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
>   at 
> java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
>   at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
>   at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:546)
>   at 
> java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
>   at 
> java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:505)
>   at 
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.upgrade(UpgradeTestBase.java:92)
>   at 
> org.apache.cassandra.distributed.upgrade.UpgradeTest.upgradeTest(UpgradeTest.java:35)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15463) Fix in-jvm dtest java 11 compatibility

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15463:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Fix in-jvm dtest java 11 compatibility
> --
>
> Key: CASSANDRA-15463
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15463
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0, 4.0-alpha3
>
>
> The url classloader used by the in jvm dtests is not accessible by default in 
> java 11.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15489) Allow dtest jar directory to be configurable

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15489:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

>  Allow dtest jar directory to be configurable
> -
>
> Key: CASSANDRA-15489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15489
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Doug Rohrer
>Assignee: Doug Rohrer
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha3
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In some circumstances, we may want to use a non-hard-coded directory as the 
> source for dtest jars. We should allow for a system property to change the 
> default `build` directory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15505) Add message interceptors to in-jvm dtests

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15505:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Add message interceptors to in-jvm dtests
> -
>
> Key: CASSANDRA-15505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15505
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha4
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Currently we only have means to filter messages in in-jvm tests. We need a 
> facility to intercept and modify the messages between nodes for testing 
> purposes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15525) Fix flakey test - org.apache.cassandra.distributed.test.RepairDigestTrackingTest testPurgeableTombstonesAreIgnore

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15525:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Fix flakey test - 
> org.apache.cassandra.distributed.test.RepairDigestTrackingTest  
> testPurgeableTombstonesAreIgnore
> --
>
> Key: CASSANDRA-15525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15525
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Priority: Normal
>
> {code}
> junit.framework.AssertionFailedError: Forked Java VM exited abnormally. 
> Please note the time in the report does not reflect the time until the VM 
> exit.
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> Here are the logs
> {code}
> [junit-timeout] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC 
> was deprecated in version 9.0 and will likely be removed in a future release.
> [junit-timeout] Testsuite: 
> org.apache.cassandra.distributed.test.RepairDigestTrackingTest
> [junit-timeout] #
> [junit-timeout] # A fatal error has been detected by the Java Runtime 
> Environment:
> [junit-timeout] #
> [junit-timeout] #  SIGSEGV (0xb) at pc=0x7f81932fa25f, pid=2628, tid=2635
> [junit-timeout] #
> [junit-timeout] # JRE version: OpenJDK Runtime Environment (11.0.5+10) (build 
> 11.0.5+10-LTS)
> [junit-timeout] # Java VM: OpenJDK 64-Bit Server VM (11.0.5+10-LTS, mixed 
> mode, tiered, compressed oops, concurrent mark sweep gc, linux-amd64)
> [junit-timeout] # Problematic frame:
> [junit-timeout] # V  [libjvm.so+0x48225f]  
> MarkAndPushClosure::do_oop(oopDesc**)+0xaf
> [junit-timeout] #
> [junit-timeout] # Core dump will be written. Default location: 
> /var/crash/core.%e.%p
> [junit-timeout] #
> [junit-timeout] # An error report file with more information is saved as:
> [junit-timeout] # hs_err_pid2628.log
> [junit-timeout] #
> [junit-timeout] # If you would like to submit a bug report, please visit:
> [junit-timeout] #   http://bugreport.java.com/bugreport/crash.jsp
> [junit-timeout] #
> [junit-timeout] Testsuite: 
> org.apache.cassandra.distributed.test.RepairDigestTrackingTest
> [junit-timeout] Testsuite: 
> org.apache.cassandra.distributed.test.RepairDigestTrackingTest Tests run: 1, 
> Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> org.apache.cassandra.distributed.test.RepairDigestTrackingTest:testPurgeableTombstonesAreIgnored:
>Caused an ERROR
> [junit-timeout] Forked Java VM exited abnormally. Please note the time in the 
> report does not reflect the time until the VM exit.
> [junit-timeout] junit.framework.AssertionFailedError: Forked Java VM exited 
> abnormally. Please note the time in the report does not reflect the time 
> until the VM exit.
> [junit-timeout]   at 
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> [junit-timeout]   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]   at java.base/java.util.Vector.forEach(Vector.java:1387)
> [junit-timeout]   at 
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> [junit-timeout]   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout]   at 
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> [junit-timeout]   at 
> 

[jira] [Updated] (CASSANDRA-15524) Fix flakey test - org.apache.cassandra.distributed.test.MessageForwardingTest mutationsForwardedToAllReplicasTest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15524:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Fix flakey test - org.apache.cassandra.distributed.test.MessageForwardingTest 
> mutationsForwardedToAllReplicasTest
> -
>
> Key: CASSANDRA-15524
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15524
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0, 4.0-alpha3
>
>
> {code}
> junit.framework.AssertionFailedError: /127.0.0.2 should have been randomized 
> to forward messages
>   at 
> org.apache.cassandra.distributed.test.MessageForwardingTest.lambda$mutationsForwardedToAllReplicasTest$7(MessageForwardingTest.java:87)
>   at java.base/java.util.HashMap.forEach(HashMap.java:1336)
>   at 
> org.apache.cassandra.distributed.test.MessageForwardingTest.mutationsForwardedToAllReplicasTest(MessageForwardingTest.java:87)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15506) Run in-jvm upgrade dtests in circleci

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15506:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Run in-jvm upgrade dtests in circleci
> -
>
> Key: CASSANDRA-15506
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15506
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI, Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>  Labels: CI
> Fix For: 3.0.20, 3.11.6, 4.0, 4.0-alpha3
>
>
> We should run the in-jvm upgrade dtests in circleci



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message serialization between versions

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15515:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Fix in-jvm message serialization between versions
> -
>
> Key: CASSANDRA-15515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0, 4.0-alpha3
>
>
> trunk in-jvm-dtests currently uses MessagingService.current_version to 
> serialize messages to other nodes which is wrong in mixed mode



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15509) In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15509:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
> --
>
> Key: CASSANDRA-15509
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15509
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0, 4.0-alpha3
>
>
> for example:
> https://circleci.com/gh/krummas/cassandra/2686



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15568) Message filtering should apply on the inboundSink in In-JVM dtest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15568:

Component/s: Test/dtest/java

> Message filtering should apply on the inboundSink in In-JVM dtest
> -
>
> Key: CASSANDRA-15568
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15568
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The message filtering mechanism in the in-jvm dtest helps to simulate network 
> partition/delay. 
> The problem of the current approach that adds all filters to the 
> {{MessagingService#outboundSink}} is that a blocking filter blocks the 
> following filters to be evaluated since there is only a single thread that 
> evaluates them. It further blocks the other outing messages. The typical 
> internode messaging pattern is that the coordinator node sends out multiple 
> messages to other nodes upon receiving a query. The described blocking 
> messages can happen quite often.
> The problem can be solved by moving the message filtering to the 
> {{MessagingService#inboundSink}}, so that each inbounding message is 
> naturally filtered in parallel.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15539) Extract in-jvm API and tests out of Cassandra and into a separate repository

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15539:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Extract in-jvm API and tests out of Cassandra and into a separate repository
> 
>
> Key: CASSANDRA-15539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15539
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 2.2.17, 3.0.21, 3.11.7, 4.0, 4.0-alpha4
>
>  Time Spent: 13h 20m
>  Remaining Estimate: 0h
>
> Extract in-jvm DTest _API_ and tests into a separate repository that is 
> shared between Cassandra branches. Tests themselves should be buildable using 
> just API, which is not  the case now, since cluster creation relies on impl 
> package, since we do not have factories / constructors in API.
> Main goals we’re trying to achieve:
> 1. We should be able to fail a build on API incompatibility between versions 
> 2. Make it as easy as possible to detect break APIs between versions. 
> 3. Make development of _tests_ based on in-jvm framework simpler
> 4. Reduce surface area of impact when making modifications to tests 
> Potentially, we’d also like to use a plugin to detect API incompatibilities 
> between in-jvm DTest API and in-branch implementations, and start running 
> tests using shared in-jvm test repository with each existing implementation 
> in the branch. This entails both running tests for all branches whenever 
> there’s a change in tests jar and running tests for a specific branch 
> whenever the branch has changed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15537) 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15537:

Component/s: Test/dtest/java

> 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test
> -
>
> Key: CASSANDRA-15537
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15537
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> Execution of upgrade and diff tests via cassandra-diff have proven to be one 
> of the most effective approaches toward identifying issues with the local 
> read/write path. These include instances of data loss, data corruption, data 
> resurrection, incorrect responses to queries, incomplete responses, and 
> others. Upgrade and diff tests can be executed concurrent with fault 
> injection (such as host or network failure); as well as during mixed-version 
> scenarios (such as upgrading half of the instances in a cluster, and running 
> upgradesstables on only half of the upgraded instances).
> Upgrade and diff tests are expected to continue through the release cycle, 
> and are a great way for contributors to gain confidence in the correctness of 
> the database under their own workloads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15542) In JVM test for repairs on token boundaries

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15542:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> In JVM test for repairs on token boundaries 
> 
>
> Key: CASSANDRA-15542
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15542
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Test/dtest/java
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Low
>  Labels: pull-request-available
> Fix For: 4.0-beta2
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Putting partitions on each token range +-1 and making sure the logic end to 
> end with repairs correctly handle inclusive and exclusivity of the bounds.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15587) 4.0 quality testing: Platforms and Runtimes

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15587:

Component/s: Test/dtest/java

> 4.0 quality testing: Platforms and Runtimes
> ---
>
> Key: CASSANDRA-15587
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15587
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: {color:#ff}NONE{color}*
> CASSANDRA-9608 introduces support for Java 11. We'll want to verify that 
> Cassandra under Java 11 meets expectations of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15538) 4.0 quality testing: Local Read/Write Path: Other Areas

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15538:

Component/s: Test/dtest/java

> 4.0 quality testing: Local Read/Write Path: Other Areas
> ---
>
> Key: CASSANDRA-15538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15538
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Aleksey Yeschenko*
> Testing in this area refers to the local read/write path (StorageProxy, 
> ColumnFamilyStore, Memtable, SSTable reading/writing, etc). We are still 
> finding numerous bugs and issues with the 3.0 storage engine rewrite 
> (CASSANDRA-8099). For 4.0 we want to ensure that we thoroughly cover the 
> local read/write path with techniques such as property-based testing, fuzzing 
> ([example|http://cassandra.apache.org/blog/2018/10/17/finding_bugs_with_property_based_testing.html]),
>  and a source audit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15588:

Component/s: Test/dtest/java

> 4.0 quality testing: Cluster Upgrade
> 
>
> Key: CASSANDRA-15588
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15588
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Sean McCarthy
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We've historically had numerous bugs concerning upgrading clusters from one 
> version to the other. Let's establish the supported upgrade path and ensure 
> that users can safely perform the upgrades in production.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15543:

Component/s: Test/dtest/java

> flaky test 
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0, 4.0-alpha4
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
>   at 
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15624) Avoid lazy initializing shut down instances when trying to send them messages

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15624:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Avoid lazy initializing shut down instances when trying to send them messages
> -
>
> Key: CASSANDRA-15624
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15624
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-rc
>
>
> We currently use {{to.broadcastAddressAndPort()}} when figuring out if we 
> should send a message to an instance, if that instance has been shut down it 
> will get re-initialized but not startup:ed which makes the tests fail.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15631) Add AssertJ test dependency

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15631:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Add AssertJ test dependency
> ---
>
> Key: CASSANDRA-15631
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15631
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java, Test/unit
>Reporter: Kevin Gallardo
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0, 4.0-alpha4
>
>
> See 
> [proposal|https://lists.apache.org/thread.html/rc562ec47578d0ae6f346ba9e3d7469c1cd3f8b521a72ddcb2accc47b%40%3Cdev.cassandra.apache.org%3E].
> The goal is to add [AssertJ|https://assertj.github.io/doc/] to the test 
> framework to allow for more comprehensible and easier to write tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15632) Create a deterministic in-jvm equivalent of dtest TestBootstrap.test_resumable_bootstrap

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15632:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Create a deterministic in-jvm equivalent of dtest 
> TestBootstrap.test_resumable_bootstrap
> 
>
> Key: CASSANDRA-15632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Priority: Normal
>
> This ticket is a followup on CASSANDRA-15614 - fix flakey test 
> TestBootstrap.test_resumable_bootstrap.
> The dtest is not really fully deterministic.  In-jvm test might give a way 
> for better control on the streaming failures.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15602) Separate dtest user API from cluster implementation API

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15602:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Separate dtest user API from cluster implementation API
> ---
>
> Key: CASSANDRA-15602
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15602
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Priority: Normal
>
> The Cluster and Instance interfaces are shared by the implementation and the 
> users, this leads to situations where methods like Instance.startup(Cluster) 
> should not be called by users.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15683) Fix NPE in SimpleReadWriteTest after test framework behavior change

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15683:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> Fix NPE in SimpleReadWriteTest after test framework behavior change
> ---
>
> Key: CASSANDRA-15683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15683
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Kevin Gallardo
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0, 4.0-alpha4
>
>
> In JVM dtests, all exceptions thrown in a request execution used to be always 
> wrapped in a RuntimeException.
> After CASSANDRA-15650 changes, the behavior has been changed to: If the 
> exception is a RuntimeException, just rethrow it, otherwise wrap in 
> RuntimeException, as you can see in: 
> [https://github.com/apache/cassandra/commit/dfc279a22a5563ac7a832a586914d5410426e9b7#diff-0b019281b7e97248577c82af0e663ef4R211]
> This causes the tests that were always extracting the cause from the wrapping 
> RuntimeException before, to check the root cause of the error, to throw a NPE 
> when they call {{getCause()}}, tests such as 
> {{SimpleReadWriteTest#readWithSchemaDisagreement}} and 
> {{SimpleReadWriteTest#writeWithSchemaDisagreement}}.
> Can be fixed by simply not unwrapping the cause in those tests, use the 
> thrown exception directly, if the behavior of "not always wrapping in 
> RuntimeException" is agreed to be correct.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15684) CASSANDRA-15650 was merged after dtest refactor and modified classes no longer in the project

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15684:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> CASSANDRA-15650 was merged after dtest refactor and modified classes no 
> longer in the project
> -
>
> Key: CASSANDRA-15684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15684
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 4.0-alpha4
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> CASSANDRA-15650 was based off commits before CASSANDRA-15539 which removed 
> some of the files modified in CASSANDRA-15650.  The tests were passing 
> pre-merge but off earlier commits.  On commit they started failing since the 
> dtest API no longer match so produces the following exception
> {code}
> [junit-timeout] 
> org.apache.cassandra.distributed.api.NodeToolResult$Asserts.errorContains([Ljava/lang/String;)Lorg/apache/cassandra/distributed/api/NodeToolResult$Asserts;
> [junit-timeout] java.lang.NoSuchMethodError: 
> org.apache.cassandra.distributed.api.NodeToolResult$Asserts.errorContains([Ljava/lang/String;)Lorg/apache/cassandra/distributed/api/NodeToolResult$Asserts;
> [junit-timeout] at 
> org.apache.cassandra.distributed.test.RepairCoordinatorFast.lambda$unknownHost$5(RepairCoordinatorFast.java:216)
> [junit-timeout] at 
> org.apache.cassandra.utils.AssertUtil.lambda$assertTimeoutPreemptively$0(AssertUtil.java:39)
> [junit-timeout] at 
> org.apache.cassandra.utils.AssertUtil.lambda$assertTimeoutPreemptively$1(AssertUtil.java:67)
> [junit-timeout] at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [junit-timeout] at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [junit-timeout] at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [junit-timeout] at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [junit-timeout] at java.lang.Thread.run(Thread.java:748)
> {code}
> Root cause was 4 files exited which should have been deleted in 
> CASSANDRA-15539.  Since they were not when CASSANDRA-15650 modified one it 
> didn't cause a merge conflict, but when the test runs it conflicts and fails.
> [Circle 
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15684]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15676) flaky test testWriteUnknownResult- org.apache.cassandra.distributed.test.CasWriteTest

[Alex Petrov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-15676:

Component/s: (was: Test/dtest/python)
 Test/dtest/java

> flaky test testWriteUnknownResult- 
> org.apache.cassandra.distributed.test.CasWriteTest
> -
>
> Key: CASSANDRA-15676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15676
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java
>Reporter: Kevin Gallardo
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: Screen Shot 2020-05-07 at 7.25.19 PM.png
>
>
> Failure observed in: 
> https://app.circleci.com/pipelines/github/newkek/cassandra/33/workflows/54007cf7-4424-4ec1-9655-665f6044e6d1/jobs/187/tests
> {noformat}
> testWriteUnknownResult - org.apache.cassandra.distributed.test.CasWriteTest
> junit.framework.AssertionFailedError: Expecting cause to be 
> CasWriteUncertainException
>   at 
> org.apache.cassandra.distributed.test.CasWriteTest.testWriteUnknownResult(CasWriteTest.java:257)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org